a0da72a5c9
* Add AGENTS.md with requirement to disclose agent authorship. * Check for edits to AGENTS.md * Add PR label on detected AI authorship. This PR adds an automated check if a contributor used an AI Agent to generate code. It does this by: - Adding an AGENT.md requesting the Agent to add a "co-authored by" line to each commit message it generates. - An added workflow (`llm_check.yml`) runs to check for any commit having such a matched line in it. - If it finds it, it adds the `AI/LLM` label. The label IS NOT removed automatically if a force push changes the commit message. The additional `get_real_pr_shas.yml` is a requirement sadly. It is currently impossible to get the commit the PR is based on via Github APIs. The variables `github.base_ref` or `github.pull_request.base.sha` either return the commit hash when the PR was created (breaks after rebase) or the new HEAD of the base branch after something unrelated was merged (see: https://github.com/actions/runner/issues/1689). This makes it effectively impossible to get the output of `git log actual_pr_base..HEAD` via the Github API.
82 lines
2.3 KiB
YAML
82 lines
2.3 KiB
YAML
# SECURITY:
|
|
#
|
|
# This workflow runs on pull_request_target and has write privileges.
|
|
# Those are used to add labels to the PR.
|
|
#
|
|
# **IF a user can run code in here, they would be able to extract our secrets.**
|
|
#
|
|
# This is why it doesn't run external scripts.
|
|
# The exception is the get_real_pr_shas.yml workflow.
|
|
# The input of it is santized.
|
|
|
|
name: AI/LLM Checks
|
|
|
|
on:
|
|
pull_request_target:
|
|
|
|
jobs:
|
|
real_pr_shas:
|
|
uses: rizinorg/rizin/.github/workflows/get_real_pr_shas.yml@dev
|
|
|
|
ai_checks:
|
|
name: LLM checks
|
|
permissions:
|
|
pull-requests: write
|
|
runs-on: ubuntu-latest
|
|
needs: real_pr_shas
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0 # Full history needed for commit scanning
|
|
|
|
- name: Validate input hashes
|
|
env:
|
|
BASE_SHA: ${{ needs.real_pr_shas.outputs.BASE_SHA }}
|
|
HEAD_SHA: ${{ needs.real_pr_shas.outputs.HEAD_SHA }}
|
|
run: |
|
|
if echo "$BASE_SHA" | grep "[^a-f0-9]"; then
|
|
echo "BASE_SHA is malformed: $BASE_SHA"
|
|
exit 1
|
|
fi
|
|
if echo "$HEAD_SHA" | grep "[^a-f0-9]"; then
|
|
echo "HEAD_SHA is malformed: $HEAD_SHA"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Check AGENT.md is unchanged
|
|
env:
|
|
BASE_SHA: ${{ needs.real_pr_shas.outputs.BASE_SHA }}
|
|
run: |
|
|
diff=$(git diff --name-status "$BASE_SHA" AGENTS.md)
|
|
if [[ $? -eq 128 ]]; then
|
|
echo "Failed to diff"
|
|
exit 1
|
|
fi
|
|
|
|
if [[ -n "$diff" ]]; then
|
|
echo "Edits to 'AGENT.md' are not allowed!"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Check for AI usage and label
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
REPO: ${{ github.repository }}
|
|
NUMBER: ${{ github.event.number }}
|
|
BASE_SHA: ${{ needs.real_pr_shas.outputs.BASE_SHA }}
|
|
HEAD_SHA: ${{ needs.real_pr_shas.outputs.HEAD_SHA }}
|
|
run: |
|
|
LABEL_NAME="AI/LLM"
|
|
NEEDLE="Co-authored-by agent"
|
|
|
|
diff=$(git log --pretty=%B "$BASE_SHA".."$HEAD_SHA" )
|
|
if [[ $? -eq 128 ]]; then
|
|
echo "Failed to get log"
|
|
exit 2
|
|
fi
|
|
|
|
if echo "$diff" | grep "$NEEDLE"; then
|
|
echo "Authored by AI agent"
|
|
gh pr --repo $REPO edit $NUMBER --add-label "$LABEL_NAME"
|
|
fi
|