Compare commits

...

No commits in common. "dp" and "fp" have entirely different histories.
dp ... fp

4511 changed files with 1088334 additions and 329092 deletions

View File

@ -1,18 +0,0 @@
**/.git
.DS_Store
*.dylib
*.DSYM
*.d
*.pyc
*.pyo
.*.swp
.*.swo
.*.un~
*.tmp
*.o
*.o-*
*.os
*.os-*
venv/
.venv/

View File

@ -1,11 +0,0 @@
root = true
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
[*.{py,pyx,pxd}]
charset = utf-8
indent_style = space
indent_size = 2

View File

@ -1,47 +0,0 @@
name: Bug report
description: For issues with running openpilot on your comma device
labels: ["bug"]
body:
- type: markdown
attributes:
value: >
Before creating a **bug report**, please check the following:
* If the issue likely only affects your car model or make, go back and open a **car bug report** instead.
* If the issue is related to the driving or driver monitoring models, you should open a [discussion](https://github.com/commaai/openpilot/discussions/categories/model-feedback) instead.
* Ensure you're running the latest openpilot release.
* Ensure you're using officially supported hardware. Issues running on PCs have a different issue template.
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
- type: textarea
attributes:
label: Describe the bug
description: Also include a description of how to reproduce the bug
validations:
required: true
- type: input
id: route
attributes:
label: Provide a route where the issue occurs
description: Ensure the route is fully uploaded at https://useradmin.comma.ai. We cannot look into issues without routes, or at least a Dongle ID.
placeholder: 77611a1fac303767|2020-05-11--16-37-07
validations:
required: true
- type: input
id: version
attributes:
label: openpilot version
description: If you're not on release, provide the commit hash
placeholder: 0.8.10
validations:
required: true
- type: textarea
attributes:
label: Additional info

View File

@ -1,14 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: Car bug report
url: https://github.com/commaai/opendbc/issues/new
about: For issues with a particular car make or model
- name: Join the Discord
url: https://discord.comma.ai
about: The community Discord is for both openpilot development and experience discussion
- name: Report driving behavior feedback
url: https://discord.com/channels/469524606043160576/1254834193066623017
about: Feedback for the driving and driver monitoring models goes in the #driving-feedback in Discord
- name: Community Wiki
url: https://github.com/commaai/openpilot/wiki
about: Check out our community wiki

View File

@ -1,8 +0,0 @@
---
name: Enhancement
about: For openpilot enhancement suggestions
title: ''
labels: 'enhancement'
assignees: ''
---

View File

@ -1,42 +0,0 @@
name: PC bug report
description: For issues with running openpilot on PC
labels: ["PC"]
body:
- type: markdown
attributes:
value: >
Before creating a **bug report**, please check the following:
* Ensure you're running the latest openpilot release.
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
- type: textarea
attributes:
label: Describe the bug
description: Also include a description of how to reproduce the bug
validations:
required: true
- type: input
id: os-version
attributes:
label: OS Version
placeholder: Ubuntu 24.04
validations:
required: true
- type: input
id: version
attributes:
label: openpilot version or commit
placeholder: bd36f2ec8d3559909678eff2690c10a520938367
validations:
required: false
- type: textarea
attributes:
label: Additional info

27
.github/labeler.yaml vendored
View File

@ -1,27 +0,0 @@
CI / testing:
- changed-files:
- any-glob-to-all-files: "{.github/**,**/test_*,**/test/**,Jenkinsfile}"
car:
- changed-files:
- any-glob-to-all-files: '{selfdrive/car/**,opendbc_repo}'
simulation:
- changed-files:
- any-glob-to-all-files: 'tools/sim/**'
ui:
- changed-files:
- any-glob-to-all-files: '{selfdrive/ui/**,system/ui/**}'
tools:
- changed-files:
- any-glob-to-all-files: 'tools/**'
multilanguage:
- changed-files:
- any-glob-to-all-files: 'selfdrive/ui/translations/**'
autonomy:
- changed-files:
- any-glob-to-all-files: "{selfdrive/modeld/models/**,selfdrive/test/process_replay/model_replay_ref_commit}"

View File

@ -1,68 +0,0 @@
<!-- Please copy and paste the relevant template -->
<!--- ***** Template: Fingerprint *****
**Car**
Which car (make, model, year) this fingerprint is for
**Route**
A route with the fingerprint
-->
<!--- ***** Template: Car Bugfix *****
**Description**
A description of the bug and the fix. Also link the issue if it exists.
**Verification**
Explain how you tested this bug fix.
**Route**
Route: [a route with the bug fix]
-->
<!--- ***** Template: Bugfix *****
**Description**
A description of the bug and the fix. Also link the issue if it exists.
**Verification**
Explain how you tested this bug fix.
-->
<!--- ***** Template: Car Port *****
**Checklist**
- [ ] added entry to CAR in selfdrive/car/*/values.py and ran `selfdrive/car/docs.py` to generate new docs
- [ ] test route added to [routes.py](https://github.com/commaai/openpilot/blob/master/selfdrive/car/tests/routes.py)
- [ ] route with openpilot:
- [ ] route with stock system:
- [ ] car harness used (if comma doesn't sell it, put N/A):
-->
<!--- ***** Template: Refactor *****
**Description**
A description of the refactor, including the goals it accomplishes.
**Verification**
Explain how you tested the refactor for regressions.
-->

277
.github/workflows/compile_frogpilot.yaml vendored Normal file
View File

@ -0,0 +1,277 @@
name: Compile FrogPilot
on:
workflow_dispatch:
inputs:
runner:
description: "Select the runner"
required: true
default: "c3"
type: choice
options:
- c3
- c3x
not_vetted:
description: "This branch is not vetted"
required: false
default: "false"
type: boolean
publish_frogpilot:
description: "Push to FrogPilot"
required: false
default: "false"
type: boolean
publish_staging:
description: "Push to FrogPilot-Staging"
required: false
default: "false"
type: boolean
publish_testing:
description: "Push to FrogPilot-Testing"
required: false
default: "false"
type: boolean
publish_custom_branch:
description: "Push to custom branch:"
required: false
default: ""
type: string
update_translations:
description: "Update missing/outdated translations"
required: false
default: "false"
type: boolean
vet_existing_translations:
description: "Vet existing translations"
required: false
default: "false"
type: boolean
env:
BASEDIR: "${{ github.workspace }}"
BUILD_DIR: /data/openpilot
OPENAI_API_KEY: "${{ secrets.OPENAI_API_KEY }}"
jobs:
get_branch:
runs-on:
- self-hosted
- ${{ github.event.inputs.runner }}
outputs:
branch: ${{ steps.get_branch.outputs.branch }}
python_version: ${{ steps.get_python_version.outputs.python_version }}
steps:
- name: Determine Current Branch on Runner
id: get_branch
run: |
cd $BUILD_DIR
BRANCH=$(git rev-parse --abbrev-ref HEAD)
echo "branch=$BRANCH" >> $GITHUB_OUTPUT
- name: Get Python Version from Runner
id: get_python_version
run: |
PYTHON_VERSION=$(tr -d '[:space:]' < "$BUILD_DIR/.python-version")
echo "python_version=$PYTHON_VERSION" >> $GITHUB_OUTPUT
translate:
if: inputs.update_translations
needs: get_branch
runs-on: ubuntu-latest
steps:
- name: Configure Git Identity
run: |
git config --global user.name "James"
git config --global user.email "91348155+FrogAi@users.noreply.github.com"
- name: Checkout Required Files
uses: actions/checkout@v4
with:
ref: ${{ needs.get_branch.outputs.branch }}
sparse-checkout: |
frogpilot/ui/
selfdrive/controls/lib/alerts_offroad.json
selfdrive/ui/
selfdrive/ui/update_translations.py
selfdrive/ui/translations/
selfdrive/ui/translations/auto_translate.py
- name: Set Up Python
uses: actions/setup-python@v4
with:
python-version: "${{ needs.get_branch.outputs.python_version }}"
- name: Install Dependencies
run: pip install requests
- name: Install Qt5 Tools
run: sudo apt update && sudo apt install -y qttools5-dev-tools
- name: Update Translations
run: python selfdrive/ui/update_translations.py --vanish
- name: Update Missing Translations
continue-on-error: true
run: python selfdrive/ui/translations/auto_translate.py --all-files
timeout-minutes: 300
- name: Vet Existing Translations
if: github.event.inputs.vet_existing_translations == 'true'
continue-on-error: true
run: python selfdrive/ui/translations/auto_translate.py --all-files --vet-translations
timeout-minutes: 300
- name: Commit and Push Translation Updates
run: |
if git diff --quiet selfdrive/ui/translations/*.ts; then
echo "No translation updates detected."
exit 0
fi
git fetch --unshallow origin ${{ needs.get_branch.outputs.branch }}
git checkout ${{ needs.get_branch.outputs.branch }}
git add selfdrive/ui/translations/*.ts
git commit --amend --no-edit
git push --force origin ${{ needs.get_branch.outputs.branch }}
build_and_push:
needs:
- get_branch
- translate
if: always()
runs-on:
- self-hosted
- ${{ github.event.inputs.runner }}
permissions:
contents: write
defaults:
run:
working-directory: ${{ env.BUILD_DIR }}
steps:
- name: Configure Git Identity
run: |
git config --global http.postBuffer 104857600
git config --global user.name "James"
git config --global user.email "91348155+FrogAi@users.noreply.github.com"
- name: Update Repository
run: |
git remote set-url origin https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/FrogAi/FrogPilot.git
if [ "${{ github.event.inputs.update_translations }}" = "true" ]; then
git fetch origin ${{ needs.get_branch.outputs.branch }}
git reset --hard FETCH_HEAD
fi
- name: Take Ownership of Build
run: |
sudo chown -R $(whoami):$(whoami) .
- name: Finalize Build
run: |
rm -f .clang-tidy
rm -f .dockerignore
rm -f .editorconfig
rm -f .gitattributes
rm -f .gitmodules
rm -f .lfsconfig
rm -f .overlay_init
rm -f .pre-commit-config.yaml
rm -f .sconsign.dblite
rm -f codecov.yml
rm -f conftest.py
rm -f poetry.lock
rm -f pyproject.toml
rm -f teleoprtc
rm -f Dockerfile.openpilot
rm -f Dockerfile.openpilot_base
rm -f Jenkinsfile
rm -f panda/board/obj/.placeholder
rm -f panda/board/obj/bootstub.panda.elf
rm -f panda/board/obj/bootstub.panda_h7.elf
rm -f panda/board/obj/panda.bin
rm -f panda/board/obj/panda.elf
rm -f panda/board/obj/panda_h7.bin
rm -f panda/board/obj/panda_h7.elf
rm -f panda/board/obj/version
find . -name '*.a' -delete
find . -name '*.o' -delete
find . -name '*.onnx' -delete
find . -name '*.os' -delete
find . -name '*.pyc' -delete
find . -name 'moc_*' -delete
rm -rf .devcontainer/
rm -rf .vscode/
rm -rf body/
rm -rf opendbc/generator/
rm -rf release/
rm -rf scripts/
rm -rf site_scons/
rm -rf teleoprtc_repo/
find .github -mindepth 1 -maxdepth 1 ! -name 'workflows' -exec rm -rf {} +
find .github/workflows -mindepth 1 ! \( \
-type f \( \
-name 'compile_frogpilot.yaml' -o \
-name 'review_pull_request.yaml' -o \
-name 'schedule_update.yaml' -o \
-name 'update_pr_branch.yaml' -o \
-name 'update_release_branch.yaml' -o \
-name 'update_tinygrad.yaml' \
\) \
\) -exec rm -rf {} +
find panda/board/jungle -type f ! -name '__init__.py' -delete
find panda/board/jungle -type d -empty -delete
find third_party/ -name '*x86*' -exec rm -rf {} +
find third_party/ -name '*Darwin*' -exec rm -rf {} +
find tools/ -mindepth 1 -maxdepth 1 ! \( -name '__init__.py' -o -name 'bodyteleop' -o -name 'lib' -o -name 'scripts' \) -exec rm -rf {} +
find . -name 'SConstruct' -delete
find . -name 'SConscript' -delete
find . -type d \( -iname "debug" -o -iname "test" -o -iname "tests" \) -exec rm -rf {} +
find . -type d -empty ! -path "./.git*" -delete
find . -type d -name '__pycache__' -exec rm -rf {} +
find . -type f -regex '.*matlab.*\.md' -delete
touch prebuilt
if [ "${{ github.event.inputs.not_vetted }}" = "true" ]; then
touch not_vetted
fi
- name: Add the update_date file
if: github.event.inputs.publish_staging == 'true'
run: |
curl -fLsS https://raw.githubusercontent.com/FrogAi/FrogPilot/FrogPilot-Staging/.github/update_date -o .github/update_date || echo "No update_date found, skipping."
- name: Commit Build
run: |
git add -f .
git commit -m "Compile FrogPilot"
git push --force origin HEAD
if [ "${{ github.event.inputs.publish_frogpilot }}" = "true" ]; then
git push --force origin HEAD:"FrogPilot"
fi
if [ "${{ github.event.inputs.publish_staging }}" = "true" ]; then
git push --force origin HEAD:"FrogPilot-Staging"
fi
if [ "${{ github.event.inputs.publish_testing }}" = "true" ]; then
git push --force origin HEAD:"FrogPilot-Testing"
fi
if [ -n "${{ github.event.inputs.publish_custom_branch }}" ]; then
git push --force origin HEAD:"${{ github.event.inputs.publish_custom_branch }}"
fi

View File

@ -0,0 +1,34 @@
name: Review Pull Request
on:
pull_request_target:
types: [opened, reopened]
env:
GITHUB_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
jobs:
pr_check:
runs-on: ubuntu-latest
steps:
- name: Exit If PR Opened by FrogAi
if: ${{ github.actor == 'FrogAi' }}
run: |
echo PR opened or reopened by FrogAi. No action needed.
exit 0
- name: Close PR for Invalid Target Branch
if: ${{ github.base_ref != 'MAKE-PRS-HERE' }}
run: |
gh api repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments \
-f body="Please submit your pull request to the \"MAKE-PRS-HERE\" branch."
gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }} \
-X PATCH \
-f state="closed"
- name: Acknowledge PR for Valid Target Branch
if: ${{ github.base_ref == 'MAKE-PRS-HERE' }}
run: |
gh api repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments \
-f body="Thank you for your PR! If you're not already in the FrogPilot Discord, [feel free to join](https://discord.FrogPilot.download) and let me know you've opened a PR!"

73
.github/workflows/schedule_update.yaml vendored Normal file
View File

@ -0,0 +1,73 @@
name: Schedule FrogPilot Update
on:
workflow_dispatch:
inputs:
scheduled_date:
description: "Enter the date to update the \"FrogPilot\" branch (YYYY-MM-DD)"
required: true
env:
BRANCH: FrogPilot-Staging
GITHUB_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
jobs:
schedule-update:
runs-on: ubuntu-latest
steps:
- name: Configure Git Identity
run: |
git config --global user.name "James"
git config --global user.email "91348155+FrogAi@users.noreply.github.com"
- name: Checkout ${{ env.BRANCH }}
uses: actions/checkout@v3
with:
persist-credentials: false
ref: ${{ env.BRANCH }}
fetch-depth: 3
- name: Schedule Update for ${{ github.event.inputs.scheduled_date }}
run: |
echo "${{ github.event.inputs.scheduled_date }}" > .github/update_date
git add .github/update_date
- name: Get Target Commit (Second Most Recent)
id: get_target
run: |
TARGET_COMMIT=$(git rev-parse HEAD~1)
AUTHOR_DATE=$(git show -s --format=%aD "$TARGET_COMMIT")
COMMITTER_DATE=$(git show -s --format=%cD "$TARGET_COMMIT")
echo "AUTHOR_DATE=$AUTHOR_DATE" >> $GITHUB_ENV
echo "COMMITTER_DATE=$COMMITTER_DATE" >> $GITHUB_ENV
echo "TARGET_COMMIT=$TARGET_COMMIT" >> $GITHUB_ENV
- name: Create Fixup Commit if Changes Exist
id: fixup_commit
run: |
if git diff --cached --quiet; then
echo "No changes to commit."
echo "has_changes=false" >> "$GITHUB_OUTPUT"
else
echo "Changes found, creating fixup commit."
git commit --fixup="${{ env.TARGET_COMMIT }}"
echo "has_changes=true" >> "$GITHUB_OUTPUT"
fi
- name: Autosquash Fixup into Target Commit
if: steps.fixup_commit.outputs.has_changes == 'true'
run: |
GIT_SEQUENCE_EDITOR=: git rebase --autosquash -i HEAD~3
- name: Restore Timestamps on Final Two Commits
if: steps.fixup_commit.outputs.has_changes == 'true'
run: |
git rebase --exec "GIT_COMMITTER_DATE='${{ env.COMMITTER_DATE }}' git commit --amend --no-edit --date='${{ env.AUTHOR_DATE }}'" HEAD~2
- name: Push Updated ${{ env.BRANCH }} Branch
if: steps.fixup_commit.outputs.has_changes == 'true'
run: |
git remote set-url origin https://x-access-token:${GITHUB_TOKEN}@github.com/${{ github.repository }}
git push origin "${{ env.BRANCH }}" --force-with-lease

88
.github/workflows/update_pr_branch.yaml vendored Normal file
View File

@ -0,0 +1,88 @@
name: Update MAKE-PRS-HERE
on:
push:
branches:
- FrogPilot-Testing
env:
GITHUB_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
SOURCE_BRANCH: FrogPilot-Testing
TARGET_BRANCH: MAKE-PRS-HERE
jobs:
update_branch:
runs-on: ubuntu-latest
steps:
- name: Configure Git Identity
run: |
git config --global user.name "James"
git config --global user.email "91348155+FrogAi@users.noreply.github.com"
- name: Checkout ${{ env.SOURCE_BRANCH }}
uses: actions/checkout@v3
with:
persist-credentials: false
fetch-depth: 0
ref: ${{ env.SOURCE_BRANCH }}
- name: Find "Compile FrogPilot" Commit in ${{ env.SOURCE_BRANCH }}
id: find_parent
run: |
COMMIT=$(git rev-list HEAD -n 1 --grep="Compile FrogPilot")
[ -z "$COMMIT" ] && echo "Compile commit not found." >&2 && exit 1
PARENT=$(git rev-list --parents -n 1 "$COMMIT" | awk '{print $2}')
[ -z "$PARENT" ] && echo "Parent commit not found." >&2 && exit 1
echo "parent_commit=$PARENT" >> "$GITHUB_OUTPUT"
echo "compile_commit=$COMMIT" >> "$GITHUB_OUTPUT"
- name: Fetch and Checkout ${{ env.TARGET_BRANCH }}
run: |
git fetch origin "${{ env.TARGET_BRANCH }}"
git checkout "${{ env.TARGET_BRANCH }}"
- name: Clean ${{ env.TARGET_BRANCH }} and Apply Updates from ${{ env.SOURCE_BRANCH }}
run: |
git rm -r --ignore-unmatch .
git clean -fdx
git checkout "${{ steps.find_parent.outputs.parent_commit }}" -- .
COMPILE_COMMIT="${{ steps.find_parent.outputs.compile_commit }}"
SOURCE_HEAD=$(git rev-parse "origin/${{ env.SOURCE_BRANCH }}")
[ "$COMPILE_COMMIT" != "$SOURCE_HEAD" ] && git cherry-pick --no-commit "${COMPILE_COMMIT}".."$SOURCE_HEAD"
rm -f .github/update_date
git add --all
- name: Commit and Push to ${{ env.TARGET_BRANCH }}
run: |
if git diff --staged --quiet; then
echo "Working tree is clean. No changes to commit."
exit 0
fi
TZ_VALUE="America/Phoenix"
day=$(TZ="$TZ_VALUE" date +"%-d")
month=$(TZ="$TZ_VALUE" date +"%B")
year=$(TZ="$TZ_VALUE" date +"%Y")
case $day in
11|12|13) s=th ;;
*) case $((day % 10)) in 1) s=st ;; 2) s=nd ;; 3) s=rd ;; *) s=th ;; esac ;;
esac
commit_message="${month} ${day}${s}, ${year} Update"
if git log -1 --pretty=%s | grep -q "$commit_message"; then
git commit --amend --no-edit
else
git commit -m "$commit_message"
fi
git remote set-url origin https://x-access-token:${GITHUB_TOKEN}@github.com/${{ github.repository }}
git push origin "${{ env.TARGET_BRANCH }}" --force

View File

@ -0,0 +1,131 @@
name: Update FrogPilot Branch
on:
schedule:
- cron: "0 18 * * 6"
env:
GITHUB_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
BRANCH_FROGPILOT: FrogPilot
BRANCH_PREVIOUS: FrogPilot-Previous
BRANCH_STAGING: FrogPilot-Staging
TZ: America/Phoenix
UPDATE_FILE: .github/update_date
jobs:
check_update:
runs-on: ubuntu-latest
outputs:
update_due: ${{ steps.check_update.outputs.update_due }}
scheduled_date: ${{ steps.check_update.outputs.scheduled_date }}
steps:
- name: Download the "update_date" File
id: download_update
run: |
curl -fLsS "https://raw.githubusercontent.com/FrogAi/FrogPilot/${{ env.BRANCH_STAGING }}/${{ env.UPDATE_FILE }}" -o update_date || touch update_date_missing
- name: Check If Update Is Due
id: check_update
run: |
if [ -f update_date_missing ]; then
echo "update_due=false" >> "$GITHUB_OUTPUT"
exit 0
fi
SCHEDULED_DATE=$(cat update_date)
CURRENT_DATE=$(TZ="${{ env.TZ }}" date +%F)
if [ "$SCHEDULED_DATE" != "$CURRENT_DATE" ]; then
echo "update_due=false" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "update_due=true" >> "$GITHUB_OUTPUT"
echo "scheduled_date=$SCHEDULED_DATE" >> "$GITHUB_OUTPUT"
update_branch:
needs: check_update
if: ${{ needs.check_update.outputs.update_due == 'true' }}
runs-on: ubuntu-latest
steps:
- name: Configure Git Identity
run: |
git config --global user.name "James"
git config --global user.email "91348155+FrogAi@users.noreply.github.com"
- name: Checkout ${{ env.BRANCH_STAGING }}
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ env.BRANCH_STAGING }}
- name: Authenticate with GITHUB_TOKEN
run: |
git remote set-url origin https://x-access-token:${GITHUB_TOKEN}@github.com/${{ github.repository }}
- name: Update README Date and Remove update file
run: |
DAY=$(TZ="${{ env.TZ }}" date +'%d' | sed 's/^0//')
case "$DAY" in
1|21|31) SUFFIX="st" ;;
2|22) SUFFIX="nd" ;;
3|23) SUFFIX="rd" ;;
*) SUFFIX="th" ;;
esac
MONTH=$(TZ="${{ env.TZ }}" date +'%B')
YEAR=$(TZ="${{ env.TZ }}" date +'%Y')
DATE="${MONTH} ${DAY}${SUFFIX}, ${YEAR}"
DATE_ESCAPED=$(printf '%s' "$DATE" | sed -E 's/ /%20/g; s/,/%2C/g')
sed -i -E "s|(Last%20Updated-)[^-)]*|\1${DATE_ESCAPED}|g" README.md
git add README.md
git rm -f "${{ env.UPDATE_FILE }}"
git commit -m "Updated README date to ${DATE}"
- name: Squash Commits
run: |
COMMIT_MSG=$(git log -1 --pretty=%B HEAD~1)
git reset --soft HEAD~2
git commit -m "$COMMIT_MSG"
- name: Rewrite Commit Dates to Noon ${{ env.TZ }}
run: |
COMMIT_DATETIME="${{ needs.check_update.outputs.scheduled_date }} 12:00"
COMMIT_PHX=$(TZ="${{ env.TZ }}" date -d "$COMMIT_DATETIME" +"%Y-%m-%dT%H:%M:%S %z")
git filter-branch --env-filter "export GIT_AUTHOR_DATE='$COMMIT_PHX'; export GIT_COMMITTER_DATE='$COMMIT_PHX'" "${{ env.BRANCH_STAGING }}"
- name: Fetch ${{ env.BRANCH_PREVIOUS }} and ${{ env.BRANCH_FROGPILOT }}
run: |
git fetch origin ${{ env.BRANCH_PREVIOUS }} ${{ env.BRANCH_FROGPILOT }}
- name: Wait Until Noon ${{ env.TZ }}
run: |
NOW=$(TZ="${{ env.TZ }}" date +%s)
TARGET=$(TZ="${{ env.TZ }}" date -d "12:00" +%s)
[ "$NOW" -lt "$TARGET" ] && sleep $((TARGET - NOW))
- name: Push ${{ env.BRANCH_STAGING }}
run: |
git push origin "${{ env.BRANCH_STAGING }}" --force
- name: Reset ${{ env.BRANCH_PREVIOUS }} to Match ${{ env.BRANCH_FROGPILOT }}
run: |
git switch "${{ env.BRANCH_PREVIOUS }}"
git reset --hard "origin/${{ env.BRANCH_FROGPILOT }}"
git push origin "${{ env.BRANCH_PREVIOUS }}" --force
- name: Reset ${{ env.BRANCH_FROGPILOT }} to Match ${{ env.BRANCH_STAGING }}
run: |
git switch "${{ env.BRANCH_FROGPILOT }}"
git reset --hard "origin/${{ env.BRANCH_STAGING }}"
git push origin "${{ env.BRANCH_FROGPILOT }}" --force

79
.github/workflows/update_tinygrad.yaml vendored Normal file
View File

@ -0,0 +1,79 @@
name: Update Tinygrad
on:
workflow_dispatch:
inputs:
runner:
description: "Select the runner"
required: true
default: "c3"
type: choice
options:
- c3
- c3x
jobs:
update_tinygrad:
name: Update Tinygrad
runs-on:
- self-hosted
- ${{ github.event.inputs.runner }}
steps:
- name: Get version
id: get_version
run: |
VERSION=$(grep -oP '^VERSION\s*=\s*"\K[^"]+' /data/openpilot/frogpilot/assets/model_manager.py)
echo "VERSION=$VERSION"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
- name: Clone GitLab repo
env:
GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}
run: |
rm -rf /data/tmp/frogpilot_tinygrad
mkdir -p /data/tmp/frogpilot_tinygrad
cd /data/tmp/frogpilot_tinygrad
git clone --depth 1 --branch Tinygrad "https://oauth2:${GITLAB_TOKEN}@gitlab.com/FrogAi/FrogPilot-Resources.git"
- name: Create Tinygrad Archive
run: |
set -euo pipefail
cd /data/openpilot
ARCHIVE_DIR="/data/tmp/frogpilot_tinygrad/FrogPilot-Resources"
ARCHIVE_NAME="Tinygrad_${{ steps.get_version.outputs.version }}.tar.gz"
TMPDIR="$(mktemp -d)"
touch "$TMPDIR/SConscript"
tar -czf "$ARCHIVE_NAME" \
--exclude="*.a" \
--exclude="*.o" \
--exclude="*.onnx" \
--exclude="*__pycache__*" \
--exclude="*tests*" \
--exclude="frogpilot/tinygrad_modeld/SConscript" \
frogpilot/tinygrad_modeld tinygrad_repo \
-C "$TMPDIR" \
--transform 's|^SConscript$|frogpilot/tinygrad_modeld/SConscript|' \
SConscript
mv "$ARCHIVE_NAME" "$ARCHIVE_DIR/"
rm -rf "$TMPDIR"
- name: Push updated Tinygrad
run: |
cd /data/tmp/frogpilot_tinygrad/FrogPilot-Resources
git config user.name "James"
git config user.email "91348155+FrogAi@users.noreply.github.com"
git add Tinygrad_*.tar.gz
git commit -m "Updated Tinygrad: ${{ steps.get_version.outputs.version }}" || echo "No changes to commit"
git push origin Tinygrad
- name: Cleanup temporary files
run: |
rm -rf /data/tmp/frogpilot_tinygrad

48
.gitignore vendored
View File

@ -10,13 +10,10 @@ venv/
.overlay_init
.overlay_consistent
.sconsign.dblite
model2.png
a.out
.hypothesis
.cache/
/docs_site/
*.mp4
*.dylib
*.DSYM
*.d
@ -36,21 +33,32 @@ a.out
*.class
*.pyxbldc
*.vcd
*.mo
*_pyx.cpp
*.qm
config.json
clcache
compile_commands.json
compare_runtime*.html
persist
selfdrive/pandad/pandad
cereal/services.h
cereal/gen
cereal/messaging/bridge
selfdrive/logcatd/logcatd
selfdrive/mapd/default_speeds_by_region.json
system/proclogd/proclogd
selfdrive/ui/translations/alerts_generated.h
selfdrive/ui/translations/tmp
selfdrive/test/longitudinal_maneuvers/out
selfdrive/car/tests/cars_dump
system/camerad/camerad
system/camerad/test/ae_gray_test
selfdrive/modeld/_modeld
selfdrive/modeld/_dmonitoringmodeld
/src/
notebooks
hyperthneed
provisioning
.coverage*
coverage.xml
@ -63,11 +71,11 @@ flycheck_*
cppcheck_report.txt
comma*.sh
selfdrive/modeld/thneed/compile
selfdrive/modeld/models/*.thneed
selfdrive/modeld/models/*.pkl
# openpilot log files
*.bz2
*.zst
build/
@ -75,25 +83,3 @@ build/
poetry.toml
Pipfile
### VisualStudioCode ###
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
!.vscode/*.code-snippets
# Local History for Visual Studio Code
.history/
# Built Visual Studio Code Extensions
*.vsix
### VisualStudioCode Patch ###
# Ignore all local history of files
.history
.ionide
# rick - keep panda_tici standalone
panda_tici/

1
.python-version Normal file
View File

@ -0,0 +1 @@
3.11.4

View File

@ -1,11 +0,0 @@
{
"recommendations": [
"ms-python.python",
"ms-vscode.cpptools",
"elagil.pre-commit-helper",
"charliermarsh.ruff",
"JamiTech.simply-blame",
"k--kato.intellij-idea-keybindings",
"trinm1709.dracula-theme-from-intellij"
]
}

85
.vscode/launch.json vendored
View File

@ -1,85 +0,0 @@
{
"version": "0.2.0",
"inputs": [
{
"id": "python_process",
"type": "pickString",
"description": "Select the process to debug",
"options": [
"selfdrive/controls/controlsd.py",
"system/timed/timed.py",
"tools/sim/run_bridge.py"
]
},
{
"id": "cpp_process",
"type": "pickString",
"description": "Select the process to debug",
"options": [
"selfdrive/ui/ui"
]
},
{
"id": "args",
"description": "Arguments to pass to the process",
"type": "promptString"
},
{
"id": "replayArg",
"type": "promptString",
"description": "Enter route or segment to replay."
}
],
"configurations": [
{
"name": "Python: openpilot Process",
"type": "debugpy",
"request": "launch",
"program": "${input:python_process}",
"console": "integratedTerminal",
"justMyCode": true,
"args": "${input:args}"
},
{
"name": "C++: openpilot Process",
"type": "cppdbg",
"request": "launch",
"program": "${workspaceFolder}/${input:cpp_process}",
"cwd": "${workspaceFolder}"
},
{
"name": "Attach LLDB to Replay drive",
"type": "lldb",
"request": "attach",
"pid": "${command:pickMyProcess}",
"initCommands": [
"script import time; time.sleep(3)"
]
},
{
"name": "Replay drive",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/opendbc/safety/tests/safety_replay/replay_drive.py",
"args": [
"${input:replayArg}"
],
"console": "integratedTerminal",
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}"
},
"subProcess": true,
"stopOnEntry": false
}
],
"compounds": [
{
"name": "Replay drive + Safety LLDB",
"configurations": [
"Replay drive",
"Attach LLDB to Replay drive"
]
}
]
}

36
.vscode/settings.json vendored
View File

@ -1,36 +0,0 @@
{
"editor.tabSize": 2,
"editor.insertSpaces": true,
"editor.renderWhitespace": "trailing",
"files.trimTrailingWhitespace": true,
"terminal.integrated.defaultProfile.linux": "dragonpilot",
"search.exclude": {
"**/.git": true,
"**/.venv": true,
"**/__pycache__": true,
"msgq_repo/": true,
"opendbc/": true,
"rednose/": true,
"rednose_repo/": true,
"openpilot/": true,
"teleoprtc_repo/": true,
"tinygrad/": true,
"tinygrad_repo/": true
},
"files.exclude": {
"**/.git": true,
"**/.venv": true,
"**/__pycache__": true
},
"python.analysis.exclude": [
"**/.git",
"**/.venv",
"**/__pycache__",
// exclude directories that should be using the symlinked version
"common/**",
"selfdrive/**",
"system/**",
"third_party/**",
"tools/**",
]
}

83
1.sh
View File

@ -1,83 +0,0 @@
#!/bin/env sh
persist_dir=/persist
target_dir=${persist_dir}/comma
# Change target dir from sunnylink to comma to make this no longer a test
# Function to remount /persist as read-only
cleanup() {
echo "Remounting ${persist_dir} as read-only..."
sudo mount -o remount,ro ${persist_dir}
}
# Function to check and backup existing keys
backup_keys() {
if [ -f "id_rsa" ] || [ -f "id_rsa.pub" ]; then
timestamp=$(date +%s)
backup_base="id_rsa_backup_$timestamp"
backup_private="$backup_base"
backup_public="${backup_base}.pub"
# Ensure we're not overwriting an existing backup
counter=0
while [ -f "$backup_private" ] || [ -f "$backup_public" ]; do
counter=$((counter + 1))
backup_private="${backup_base}_$counter"
backup_public="${backup_base}_$counter.pub"
done
# Backup the keys
cp id_rsa "$backup_private"
cp id_rsa.pub "$backup_public"
# Verify the backup
original_private_hash=$(sha256sum id_rsa | cut -d ' ' -f 1)
backup_private_hash=$(sha256sum "$backup_private" | cut -d ' ' -f 1)
original_public_hash=$(sha256sum id_rsa.pub | cut -d ' ' -f 1)
backup_public_hash=$(sha256sum "$backup_public" | cut -d ' ' -f 1)
if [ "$original_private_hash" = "$backup_private_hash" ] && [ "$original_public_hash" = "$backup_public_hash" ]; then
echo "Backup verified successfully."
# Safe to delete original keys after successful backup verification
else
echo "Backup verification failed. Aborting operation."
exit 1
fi
echo "Existing keys backed up as $backup_private and $backup_public"
fi
}
# Trap any signal that exits the script to run cleanup function
trap cleanup EXIT
# Remount /persist as read-write
sudo mount -o remount,rw ${persist_dir}
# Ensure the directory exists
mkdir -p ${target_dir}
cd ${target_dir}
# Check for and backup existing keys
#backup_keys
# Generate new keys
if ! ssh-keygen -t rsa -b 4096 -m PEM -f id_rsa -N ''; then
echo "Failed to generate new RSA keys. Exiting..."
exit 1
fi
# Convert the generated SSH public key to PEM format and store it temporarily
if ! openssl rsa -pubout -in id_rsa -out id_rsa.pub -outform PEM; then
echo "Failed to convert the public key to PEM format. Exiting..."
exit 1
fi
# Display the public key
echo "Displaying the public key:"
cat id_rsa.pub
# Cleanup will be called automatically due to trap on EXIT
#echo "Operation completed successfully. System will reboot now."
#sudo reboot

View File

@ -1,89 +0,0 @@
```mermaid
flowchart TD
B000["devel-staging"] ---> CORE["core"]
CORE ---> CORE_001["core-feat/params"]
CORE_001 ---> CORE_002["core-feat/panel"]
CORE_002 ---> MIN["min"]
MIN ---> MIN_001["min-feat/dev/c3"]
MIN ---> MIN_002["min-feat/dev/o3"]
MIN ---> MIN_003["min-feat/lat/alka"]
MIN ---> MIN_004["min-feat/ui/display-mode"]
MIN ---> MIN_005["min-feat/dev/model-selector"]
MIN ---> MIN_006["min-feat/lat/lca"]
MIN ---> MIN_007["min-feat/dev/on-off-road"]
MIN ---> MIN_008["min-feat/ui/hide-hud"]
MIN ---> MIN_009["min-feat/lon/ext-radar"]
MIN ---> MIN_010["min-feat/lat/road-edge-detection"]
MIN ---> MIN_011["min-feat/ui/rainbow-path"]
MIN ---> MIN_012["min-feat/lon/acm"]
MIN ---> MIN_013["min-feat/lon/aem"]
MIN ---> MIN_014["min-feat/lon/dtsc"]
MIN ---> MIN_015["min-feat/dev/alert-mode"]
MIN ---> MIN_016["min-feat/dev/auto-shutdown"]
MIN ---> MIN_017["min-feat/ui/lead-stats"]
MIN ---> MIN_018["min-feat/ui/border-indicator"]
MIN ---> MIN_019["min-feat/dev/dashy"]
MIN ---> MIN_010["min-feat/dev/delay-loggerd"]
MIN ---> MIN_021["min-feat/dev/disable-connect"]
MIN ---> MIN_022["min-feat/dev/tether-on-boot"]
MIN_001 ---> FULL["full"]
MIN_002 ---> FULL
MIN_003 ---> FULL
MIN_004 ---> FULL
MIN_005 ---> FULL
MIN_006 ---> FULL
MIN_007 ---> FULL
MIN_008 ---> FULL
MIN_009 ---> FULL
MIN_010 ---> FULL
MIN_011 ---> FULL
MIN_012 ---> FULL
MIN_013 ---> FULL
MIN_014 ---> FULL
MIN_015 ---> FULL
MIN_016 ---> FULL
MIN_017 ---> FULL
MIN_018 ---> FULL
MIN_019 ---> FULL
MIN_020 ---> FULL
MIN_021 ---> FULL
MIN_022 ---> FULL
FULL ---> TOYOTA_001[brand/toyota/safety-common]
FULL ---> TOYOTA_002[brand/toyota/door-auto-lock-unlock]
FULL ---> TOYOTA_003[brand/toyota/tss1-sng]
FULL ---> TOYOTA_004[brand/toyota/radar-filter]
FULL ---> TOYOTA_005[brand/toyota/sdsu]
FULL ---> TOYOTA_006[brand/toyota/dsu-bypass]
FULL ---> TOYOTA_007[brand/toyota/zss]
FULL ---> TOYOTA_008[brand/toyota/stock-lon]
FULL ---> VAG_001[brand/vag/a0-sng]
FULL ---> VAG_002[brand/vag/pq-steering-patch]
FULL ---> VAG_003[brand/vag/pq-no-dashcam]
FULL ---> VAG_004[brand/vag/avoid-eps-lockout]
FULL ---> HKG_001[brand/hkg/smdps]
FULL ---> HONDA_001[brand/honda/eps-mod]
FULL ---> SUBARU_001[brand/subaru/torque-3071]
TOYOTA_001 ---> TOYOTA[pre-toyota]
TOYOTA_002 ---> TOYOTA
TOYOTA_003 ---> TOYOTA
TOYOTA_004 ---> TOYOTA
TOYOTA_005 ---> TOYOTA
TOYOTA_006 ---> TOYOTA
TOYOTA_007 ---> TOYOTA
TOYOTA_008 ---> TOYOTA
VAG_001 ---> VAG[pre-vag]
VAG_002 ---> VAG
VAG_003 ---> VAG
VAG_004 ---> VAG
HKG_001 ---> HKG[pre-hkg]
HONDA_001 ---> HONDA[pre-honda]
SUBARU_001 ---> SUBARU[pre-subaru]
TOYOTA ---> PRE[pre]
VAG ---> PRE
HKG ---> PRE
HONDA ---> PRE
SUBARU ---> PRE
PRE ---> PRE_PATCH[pre-patch]
PRE_PATCH ---> PREBUILD[pre-build]
PREBUILD ---> VERSION[x.x.x]
```

File diff suppressed because it is too large Load Diff

View File

@ -1,14 +0,0 @@
FROM ghcr.io/commaai/openpilot-base:latest
ENV PYTHONUNBUFFERED=1
ENV OPENPILOT_PATH=/home/batman/openpilot
RUN mkdir -p ${OPENPILOT_PATH}
WORKDIR ${OPENPILOT_PATH}
COPY . ${OPENPILOT_PATH}/
ENV UV_BIN="/home/batman/.local/bin/"
ENV PATH="$UV_BIN:$PATH"
RUN UV_PROJECT_ENVIRONMENT=$VIRTUAL_ENV uv run scons --cache-readonly -j$(nproc)

View File

@ -1,81 +0,0 @@
FROM ubuntu:24.04
ENV PYTHONUNBUFFERED=1
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends sudo tzdata locales ssh pulseaudio xvfb x11-xserver-utils gnome-screenshot python3-tk python3-dev && \
rm -rf /var/lib/apt/lists/*
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en
ENV LC_ALL=en_US.UTF-8
COPY tools/install_ubuntu_dependencies.sh /tmp/tools/
RUN /tmp/tools/install_ubuntu_dependencies.sh && \
rm -rf /var/lib/apt/lists/* /tmp/* && \
cd /usr/lib/gcc/arm-none-eabi/* && \
rm -rf arm/ thumb/nofp thumb/v6* thumb/v8* thumb/v7+fp thumb/v7-r+fp.sp
# Add OpenCL
RUN apt-get update && apt-get install -y --no-install-recommends \
apt-utils \
alien \
unzip \
tar \
curl \
xz-utils \
dbus \
gcc-arm-none-eabi \
tmux \
vim \
libx11-6 \
wget \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir -p /tmp/opencl-driver-intel && \
cd /tmp/opencl-driver-intel && \
wget https://github.com/intel/llvm/releases/download/2024-WW14/oclcpuexp-2024.17.3.0.09_rel.tar.gz && \
wget https://github.com/oneapi-src/oneTBB/releases/download/v2021.12.0/oneapi-tbb-2021.12.0-lin.tgz && \
mkdir -p /opt/intel/oclcpuexp_2024.17.3.0.09_rel && \
cd /opt/intel/oclcpuexp_2024.17.3.0.09_rel && \
tar -zxvf /tmp/opencl-driver-intel/oclcpuexp-2024.17.3.0.09_rel.tar.gz && \
mkdir -p /etc/OpenCL/vendors && \
echo /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64/libintelocl.so > /etc/OpenCL/vendors/intel_expcpu.icd && \
cd /opt/intel && \
tar -zxvf /tmp/opencl-driver-intel/oneapi-tbb-2021.12.0-lin.tgz && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbb.so /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbbmalloc.so /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbb.so.12 /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbbmalloc.so.2 /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
mkdir -p /etc/ld.so.conf.d && \
echo /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 > /etc/ld.so.conf.d/libintelopenclexp.conf && \
ldconfig -f /etc/ld.so.conf.d/libintelopenclexp.conf && \
cd / && \
rm -rf /tmp/opencl-driver-intel
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=graphics,utility,compute
ENV QTWEBENGINE_DISABLE_SANDBOX=1
RUN dbus-uuidgen > /etc/machine-id
ARG USER=batman
ARG USER_UID=1001
RUN useradd -m -s /bin/bash -u $USER_UID $USER
RUN usermod -aG sudo $USER
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER $USER
COPY --chown=$USER pyproject.toml uv.lock /home/$USER
COPY --chown=$USER tools/install_python_dependencies.sh /home/$USER/tools/
ENV VIRTUAL_ENV=/home/$USER/.venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN cd /home/$USER && \
tools/install_python_dependencies.sh && \
rm -rf tools/ pyproject.toml uv.lock .cache
USER root
RUN sudo git config --global --add safe.directory /tmp/openpilot

269
Jenkinsfile vendored
View File

@ -1,269 +0,0 @@
def retryWithDelay(int maxRetries, int delay, Closure body) {
for (int i = 0; i < maxRetries; i++) {
try {
return body()
} catch (Exception e) {
sleep(delay)
}
}
throw Exception("Failed after ${maxRetries} retries")
}
def device(String ip, String step_label, String cmd) {
withCredentials([file(credentialsId: 'id_rsa', variable: 'key_file')]) {
def ssh_cmd = """
ssh -o ConnectTimeout=5 -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o BatchMode=yes -o StrictHostKeyChecking=no -i ${key_file} 'comma@${ip}' exec /usr/bin/bash <<'END'
set -e
export TERM=xterm-256color
shopt -s huponexit # kill all child processes when the shell exits
export CI=1
export PYTHONWARNINGS=error
export LOGPRINT=debug
export TEST_DIR=${env.TEST_DIR}
export SOURCE_DIR=${env.SOURCE_DIR}
export GIT_BRANCH=${env.GIT_BRANCH}
export GIT_COMMIT=${env.GIT_COMMIT}
export CI_ARTIFACTS_TOKEN=${env.CI_ARTIFACTS_TOKEN}
export GITHUB_COMMENTS_TOKEN=${env.GITHUB_COMMENTS_TOKEN}
export AZURE_TOKEN='${env.AZURE_TOKEN}'
# only use 1 thread for tici tests since most require HIL
export PYTEST_ADDOPTS="-n0 -s"
export GIT_SSH_COMMAND="ssh -i /data/gitkey"
source ~/.bash_profile
if [ -f /TICI ]; then
source /etc/profile
rm -rf /tmp/tmp*
rm -rf ~/.commacache
rm -rf /dev/shm/*
rm -rf /dev/tmp/tmp*
if ! systemctl is-active --quiet systemd-resolved; then
echo "restarting resolved"
sudo systemctl start systemd-resolved
sleep 3
fi
# restart aux USB
if [ -e /sys/bus/usb/drivers/hub/3-0:1.0 ]; then
echo "restarting aux usb"
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/unbind
sleep 0.5
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/bind
fi
fi
if [ -f /data/openpilot/launch_env.sh ]; then
source /data/openpilot/launch_env.sh
fi
ln -snf ${env.TEST_DIR} /data/pythonpath
cd ${env.TEST_DIR} || true
time ${cmd}
END"""
sh script: ssh_cmd, label: step_label
}
}
def deviceStage(String stageName, String deviceType, List extra_env, def steps) {
stage(stageName) {
if (currentBuild.result != null) {
return
}
if (isReplay()) {
error("REPLAYING TESTS IS NOT ALLOWED. FIX THEM INSTEAD.")
}
def extra = extra_env.collect { "export ${it}" }.join('\n');
def branch = env.BRANCH_NAME ?: 'master';
def gitDiff = sh returnStdout: true, script: 'curl -s -H "Authorization: Bearer ${GITHUB_COMMENTS_TOKEN}" https://api.github.com/repos/commaai/openpilot/compare/master...${GIT_BRANCH} | jq .files[].filename || echo "/"', label: 'Getting changes'
lock(resource: "", label: deviceType, inversePrecedence: true, variable: 'device_ip', quantity: 1, resourceSelectStrategy: 'random') {
docker.image('ghcr.io/commaai/alpine-ssh').inside('--user=root') {
timeout(time: 35, unit: 'MINUTES') {
retry (3) {
def date = sh(script: 'date', returnStdout: true).trim();
device(device_ip, "set time", "date -s '" + date + "'")
device(device_ip, "git checkout", extra + "\n" + readFile("selfdrive/test/setup_device_ci.sh"))
}
steps.each { item ->
def name = item[0]
def cmd = item[1]
def args = item[2]
def diffPaths = args.diffPaths ?: []
def cmdTimeout = args.timeout ?: 9999
if (branch != "master" && !branch.contains("__jenkins_loop_") && diffPaths && !hasPathChanged(gitDiff, diffPaths)) {
println "Skipping ${name}: no changes in ${diffPaths}."
return
} else {
timeout(time: cmdTimeout, unit: 'SECONDS') {
device(device_ip, name, cmd)
}
}
}
}
}
}
}
}
def hasPathChanged(String gitDiff, List<String> paths) {
for (path in paths) {
if (gitDiff.contains(path)) {
return true
}
}
return false
}
def isReplay() {
def replayClass = "org.jenkinsci.plugins.workflow.cps.replay.ReplayCause"
return currentBuild.rawBuild.getCauses().any{ cause -> cause.toString().contains(replayClass) }
}
def setupCredentials() {
withCredentials([
string(credentialsId: 'azure_token', variable: 'AZURE_TOKEN'),
]) {
env.AZURE_TOKEN = "${AZURE_TOKEN}"
}
withCredentials([
string(credentialsId: 'ci_artifacts_pat', variable: 'CI_ARTIFACTS_TOKEN'),
]) {
env.CI_ARTIFACTS_TOKEN = "${CI_ARTIFACTS_TOKEN}"
}
withCredentials([
string(credentialsId: 'post_comments_github_pat', variable: 'GITHUB_COMMENTS_TOKEN'),
]) {
env.GITHUB_COMMENTS_TOKEN = "${GITHUB_COMMENTS_TOKEN}"
}
}
def step(String name, String cmd, Map args = [:]) {
return [name, cmd, args]
}
node {
env.CI = "1"
env.PYTHONWARNINGS = "error"
env.TEST_DIR = "/data/openpilot"
env.SOURCE_DIR = "/data/openpilot_source/"
setupCredentials()
env.GIT_BRANCH = checkout(scm).GIT_BRANCH
env.GIT_COMMIT = checkout(scm).GIT_COMMIT
def excludeBranches = ['__nightly', 'devel', 'devel-staging', 'release3', 'release3-staging',
'release-tici', 'release-tizi', 'release-tizi-staging', 'testing-closet*', 'hotfix-*']
def excludeRegex = excludeBranches.join('|').replaceAll('\\*', '.*')
if (env.BRANCH_NAME != 'master' && !env.BRANCH_NAME.contains('__jenkins_loop_')) {
properties([
disableConcurrentBuilds(abortPrevious: true)
])
}
try {
if (env.BRANCH_NAME == 'devel-staging') {
deviceStage("build release-tizi-staging", "tizi-needs-can", [], [
step("build release-tizi-staging", "RELEASE_BRANCH=release-tizi-staging $SOURCE_DIR/release/build_release.sh"),
])
}
if (env.BRANCH_NAME == '__nightly') {
parallel (
'nightly': {
deviceStage("build nightly", "tizi-needs-can", [], [
step("build nightly", "RELEASE_BRANCH=nightly $SOURCE_DIR/release/build_release.sh"),
])
},
'nightly-dev': {
deviceStage("build nightly-dev", "tizi-needs-can", [], [
step("build nightly-dev", "PANDA_DEBUG_BUILD=1 RELEASE_BRANCH=nightly-dev $SOURCE_DIR/release/build_release.sh"),
])
},
)
}
if (!env.BRANCH_NAME.matches(excludeRegex)) {
parallel (
'onroad tests': {
deviceStage("onroad", "tizi-needs-can", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("check dirty", "release/check-dirty.sh"),
step("onroad tests", "pytest selfdrive/test/test_onroad.py -s", [timeout: 60]),
])
},
'HW + Unit Tests': {
deviceStage("tizi-hardware", "tizi-common", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test pandad", "pytest selfdrive/pandad/tests/test_pandad.py", [diffPaths: ["panda", "selfdrive/pandad/"]]),
step("test power draw", "pytest -s system/hardware/tici/tests/test_power_draw.py"),
step("test encoder", "LD_LIBRARY_PATH=/usr/local/lib pytest system/loggerd/tests/test_encoder.py", [diffPaths: ["system/loggerd/"]]),
step("test manager", "pytest system/manager/test/test_manager.py"),
])
},
'loopback': {
deviceStage("loopback", "tizi-loopback", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("test pandad loopback", "pytest selfdrive/pandad/tests/test_pandad_loopback.py"),
])
},
'camerad OX03C10': {
deviceStage("OX03C10", "tizi-ox03c10", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 60]),
step("test exposure", "pytest system/camerad/test/test_exposure.py"),
])
},
'camerad OS04C10': {
deviceStage("OS04C10", "tici-os04c10", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 60]),
step("test exposure", "pytest system/camerad/test/test_exposure.py"),
])
},
'sensord': {
deviceStage("LSM + MMC", "tizi-lsmc", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test sensord", "pytest system/sensord/tests/test_sensord.py"),
])
},
'replay': {
deviceStage("model-replay", "tizi-replay", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
step("model replay", "selfdrive/test/process_replay/model_replay.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
])
},
'tizi': {
deviceStage("tizi", "tizi", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("test pandad loopback", "SINGLE_PANDA=1 pytest selfdrive/pandad/tests/test_pandad_loopback.py"),
step("test pandad spi", "pytest selfdrive/pandad/tests/test_pandad_spi.py"),
step("test amp", "pytest system/hardware/tici/tests/test_amplifier.py"),
// TODO: enable once new AGNOS is available
// step("test esim", "pytest system/hardware/tici/tests/test_esim.py"),
step("test qcomgpsd", "pytest system/qcomgpsd/tests/test_qcomgpsd.py", [diffPaths: ["system/qcomgpsd/"]]),
])
},
)
}
} catch (Exception e) {
currentBuild.result = 'FAILED'
throw e
}
}

View File

@ -1,30 +0,0 @@
Copyright (c) 2019, Rick Lan
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, and/or sublicense,
for non-commercial purposes only, subject to the following conditions:
- The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
- Commercial use (e.g. use in a product, service, or activity intended to
generate revenue) is prohibited without explicit written permission from
the copyright holder.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
---
Copyright (c) 2018, Comma.ai, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

256
README.md
View File

@ -1,74 +1,252 @@
![](dragonpilot/selfdrive/assets/dragonpilot.png)
<div align="center" style="text-align: center;">
[Read this in English](README_EN.md)
<h1>openpilot</h1>
# **🐲 dragonpilot - 賦予您的愛車「龍」之魂**
<p>
<b>openpilot is an operating system for robotics.</b>
<br>
Currently, it upgrades the driver assistance system in 300+ supported cars.
</p>
**我們與您一同翱翔於更智慧、更貼心的駕駛旅程。**
<h3>
<a href="https://docs.comma.ai">Docs</a>
<span> · </span>
<a href="https://docs.comma.ai/contributing/roadmap/">Roadmap</a>
<span> · </span>
<a href="https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md">Contribute</a>
<span> · </span>
<a href="https://discord.comma.ai">Community</a>
<span> · </span>
<a href="https://comma.ai/shop">Try it on a comma 3X</a>
</h3>
## **👋 嘿, 朋友,歡迎您的到來!**
Quick start: `bash <(curl -fsSL openpilot.comma.ai)`
`dragonpilot` 誕生於 2019 年,由三位早期的 openpilot 華人玩家共同創立。初衷很簡單:為廣大的華人用戶、玩家們提供一個友善的交流環境、更簡便的設定協助,並加入更多適合在地使用的貼心功能。
[![openpilot tests](https://github.com/commaai/openpilot/actions/workflows/selfdrive_tests.yaml/badge.svg)](https://github.com/commaai/openpilot/actions/workflows/selfdrive_tests.yaml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![X Follow](https://img.shields.io/twitter/follow/comma_ai)](https://x.com/comma_ai)
[![Discord](https://img.shields.io/discord/469524606043160576)](https://discord.comma.ai)
我們深知在地化的重要性,特別是語言的親切感。因此,我們率先導入了完整的中文介面,讓 `dragonpilot` 迅速在華語地區累積了口碑,也讓華人的使用者數量在全球名列前茅。這份來自在地的支持,是我們持續前進的最大動力。
</div>
我們以功能強大的 [openpilot](https://github.com/commaai/openpilot) 為基礎——這套據美國消費者報告評測優於市售車方案的開源輔助駕駛系統——融入了更多在地化的巧思與客製化的溫度,希望能打造出最符合您需求的駕駛夥伴。(您也可以參考我們 repo 中保留的 [openpilot 原始說明檔案](README_OPENPILOT.md))
<table>
<tr>
<td><a href="https://youtu.be/NmBfgOanCyk" title="Video By Greer Viau"><img src="https://github.com/commaai/openpilot/assets/8762862/2f7112ae-f748-4f39-b617-fabd689c3772"></a></td>
<td><a href="https://youtu.be/VHKyqZ7t8Gw" title="Video By Logan LeGrand"><img src="https://github.com/commaai/openpilot/assets/8762862/92351544-2833-40d7-9e0b-7ef7ae37ec4c"></a></td>
<td><a href="https://youtu.be/SUIZYzxtMQs" title="A drive to Taco Bell"><img src="https://github.com/commaai/openpilot/assets/8762862/05ceefc5-2628-439c-a9b2-89ce77dc6f63"></a></td>
</tr>
</table>
取名 `dragonpilot`,是因為我們希望它能像神話中的「龍」一樣,既強大又充滿智慧,為您的行車安全保駕護航。龍,在我們華人文化中,更是吉祥與力量的象徵,也代表著我們的根源與驕傲。
## **✨ dragonpilot 的里程碑**
Using openpilot in a car
------
我們不僅保留了 openpilot 的核心優勢,更達成了許多從社群回饋中誕生的里程碑,這些是我們引以為傲的足跡:
To use openpilot in a car, you need four things:
1. **Supported Device:** a comma 3/3X, available at [comma.ai/shop](https://comma.ai/shop/comma-3x).
2. **Software:** The setup procedure for the comma 3/3X allows users to enter a URL for custom software. Use the URL `openpilot.comma.ai` to install the release version.
3. **Supported Car:** Ensure that you have one of [the 275+ supported cars](docs/CARS.md).
4. **Car Harness:** You will also need a [car harness](https://comma.ai/shop/car-harness) to connect your comma 3/3X to your car.
* **🚘 全時置中車道維持 (ALKA)**
We have detailed instructions for [how to install the harness and device in a car](https://comma.ai/setup). Note that it's possible to run openpilot on [other hardware](https://blog.comma.ai/self-driving-car-for-free/), although it's not plug-and-play.
這不只是一個功能,更是 `dragonpilot` 的哲學。我們最早於 [0.6.2 版本](https://github.com/dragonpilot-community/dragonpilot/blob/2861467183d62151024320447ba04d18fc3fe1e6/selfdrive/car/toyota/carstate.py#L199) 時便實現了這個功能,其開發歷程始於 2017 Lexus IS300h接著擴展至 Toyota 全車系,並逐步延伸到其他支援的品牌。它能溫柔地輔助您,讓車輛始終穩定地保持在車道中央,提供一份額外的安心與從容。
------
* **🌐 率先導入多國語言介面**
<div align="center" style="text-align: center;">
在官方 openpilot 還未支援前,我們便已將多國語言介面實現。`dragonpilot` 完整支援繁體中文、簡體中文與英文,讓操作毫無隔閡。
<h1>FrogPilot 🐸</h1>
* **💻 唯一同時支援多硬體平台**
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/FrogAi/FrogPilot)
[![Discord](https://img.shields.io/discord/1137853399715549214?label=Discord)](https://discord.frogpilot.download)
[![Last Updated](https://img.shields.io/badge/Last%20Updated-October%2018th%2C%202025-brightgreen)](https://github.com/FrogAi/FrogPilot/releases/latest)
[![Wiki](https://img.shields.io/badge/Wiki-FrogPilot-blue?logo=wiki)](https://frogpilot.wiki.gg/)
我們是唯一曾致力於讓專案同時兼容 EON、comma two、comma 3 與 Jetson 平台的社群分支,這份努力是為了服務最廣大的玩家社群。
此外,在 comma.ai 團隊於 0.10.0 版本宣布停止支持 comma 3 後,我們仍是唯一一個完整同時支援 comma 3、comma 3X 以及 O3、O3L、O3XLO3 系列為副廠硬體)的社群分支。
</div>
* **📜 曾榮獲官方認證第一大分支**
------
基於活躍的社群與功能創新,`dragonpilot` 曾一度成長為 comma ai 官方認證的第一大 openpilot 分支,這份榮耀屬於每一位參與者。
**FrogPilot** is a custom frog-themed fork of openpilot that embraces a collaborative, community-driven approach to push the project forward. It delivers bleeding-edge features and experimental improvements far ahead of official releases. As an unofficial and highly experimental version of openpilot, **FrogPilot** should always be used with caution!
## **🧑‍💻 設計理念 - 少即是多 (Less is More)**
openpilot vs **FrogPilot**
------
隨著 openpilot 的 AI 模型日益強大,許多過去需要手動微調的功能,現在都已能透過更先進的模型來實現。因此,我們現在的開發重心回歸到 **「最小化修改」(minimal changes)** 的核心原則上。
#### Community
| Feature | openpilot | **FrogPilot** |
|---------|:---------:|:---------:|
| A Welcoming Community | ❌ | ✅ |
| Erich / Primary Moderators / 🦇 | ✅ | ❌ |
我們的目標是為您提供最純粹、最接近官方的 openpilot 駕駛感受,同時保留 `dragonpilot` 那些經過時間考驗、最受社群喜愛的經典功能。我們相信,在強大的 AI 基礎上,簡潔即是力量。
#### Core Features
| Feature | openpilot | **FrogPilot** |
|---------|:---------:|:---------:|
| Always On Lateral (Steering) | ❌ | ✅ |
| Blind Spot Integration | ✅ | ✅ |
| Conditional Experimental Mode | ❌ | ✅ |
| Custom Themes | ❌ | ✅ |
| Driver Monitoring | ✅ | ✅ |
| Driving Model Selector | ❌ | ✅ |
| Holiday Themes | ❌ | ✅ |
| Speed Limit Support | ❌ | ✅ |
| Weather Detection | ❌ | ✅ |
## **🛠️ 硬件的足跡 - 一路走來的夥伴們**
#### Device & Hardware
| Feature | openpilot | **FrogPilot** |
|---------|:---------:|:---------:|
| Advanced Volume Controller | ❌ | ✅ |
| Automatic Version Backups | ❌ | ✅ |
| C3 Support | ❌ | ✅ |
| comma Pedal Support | ❌ | ✅ |
| High Quality Recordings | ❌ | ✅ |
| SDSU Support | ❌ | ✅ |
| ZSS Support | ❌ | ✅ |
從最早的 **EON**,到官方的 **comma two / three (C2/C3/C3X)**,再到社群中各式各樣充滿智慧的**副廠機 (如 C1.5, O2, O3, O3L, O3XL 等)**,甚至我們也曾探索過在 [**Jetson Xavier NX**](https://github.com/eFiniLan/xnxpilot) 上的可能性。
#### Gas/Brake
| Feature | openpilot | **FrogPilot** |
|---------|:---------:|:---------:|
| Adaptive Cruise Control (ACC) | ✅ | ✅ |
| Advanced Live Tuning | ❌ | ✅ |
| Custom Following Distances | ❌ | ✅ |
| Faster Human-Like Acceleration | ❌ | ✅ |
| Human-Like Speed Control in Curves | ❌ | ✅ |
| Smoother Human-Like Braking | ❌ | ✅ |
目前最新版本主要支援: comma3 / 3X 以及 O3 / O3L / O3XL 等社群硬體。
針對 EON / C1.5 / C2 等舊款硬體,最後支援的版本位於 [d2 分支](https://github.com/dragonpilot-community/dragonpilot/tree/d2)。
無論您手上是哪一款設備,都代表著您對開源駕駛輔助的一份熱情。
#### Steering
| Feature | openpilot | **FrogPilot** |
|---------|:---------:|:---------:|
| Advanced Live Tuning | ❌ | ✅ |
| Automatic Lane Changes | ❌ | ✅ |
| Increased Steering Torque* | ❌ | ✅ |
| Lane Centering (LKAS) | ✅ | ✅ |
| Lane Change Assist | ✅ | ✅ |
## **🫂 加入我們,成為「尋龍者」的一份子**
*Select vehicles only
`dragonpilot` 的成長,離不開每一位使用者的貢獻與回饋。我們是一個以**公開、透明**為原則的溫暖社群,希望在這裡能與所有對 openpilot / dragonpilot 有興趣的用戶分享、交流開發與使用上的經驗。
And much much more!
[**歡迎加入我們的 Facebook 社團進行交流!**](https://www.facebook.com/groups/930190251238639)
🌟 Highlight Features
------
## **❤️ 特別感謝**
### 🚗 Always On Lateral (AOL)
`dragonpilot` 從創立至今,從未打算透過 Patreon 等平台進行任何形式的募資。我們的初衷是建立一個讓大家能一起學習、一起成長的社群。It's all about fun, not money.
With **"Always On Lateral"**, lane-centering stays active whenever cruise control is on, even when you press the accelerator or brake. This means steering assist won't cut out during manual speed adjustments giving you continuous support through curves, traffic, or mountain roads!
然而,我們仍要對那些自發性支持本專案的朋友們,致上最誠摯的感謝。正是因為有您們的鼓勵,我們才有更大的動力持續前進。
---
[**我們的贊助者名單**](SPONSORS.md)
### 🧠 Conditional Experimental Mode (CEM)
### **安全聲明**
**["Experimental Mode"](https://blog.comma.ai/090release/#experimental-mode)** lets openpilot drive at the speed it thinks a human would to allow slowing for curves, stopping at stoplights/stop signs, and adapting to traffic. This makes it powerful in complex scenarios, but it's still, well, "experimental" and less predictable than **"Chill Mode"**. But **"Conditional Experimental Mode"** gives you the best of both worlds by automatically switching between **"Chill Mode"** for steady cruising and **"Experimental Mode"** for more advanced situations to help fully automate your driving experience!
`dragonpilot` 是一種駕駛**輔助**系統,並非全自動駕駛。它旨在減輕您的駕駛疲勞,提升行車安全,但駕駛人仍需時刻保持專注,並隨時準備接管車輛。請務必遵守您所在地區的交通法規。
**"Conditional Experimental Mode"** switches into **"Experimental Mode"** when conditions like these are met:
- Approaching curves and turns
- Detecting slower or stopped lead vehicles
- Driving below a set speed
- Predicting an upcoming stop (e.g. stoplight or stop sign)
**最後,再次感謝您的到來。**
Once conditions clear it returns to **"Chill Mode"** for stability and predictability.
**期待與您一同在智慧駕駛的道路上,乘「龍」而行!**
**Note: Stay attentive as "Experimental Mode" is an alpha feature and mistakes are expected!**
---
### 🎭 Driving Personalities
With **"Driving Personalities"**, you choose how the vehicle behaves with four adjustable profiles:
- **Traffic:** Catered towards stop-and-go traffic by minimizing gaps and delays
- **Aggressive:** Aimed to provide tighter following distances and quicker reactions
- **Standard:** Useful for a balanced, all-purpose driving
- **Relaxed:** A smoother driving experience with larger following distance gaps
Each profile can be fine-tuned to change the desired following distance, acceleration, and braking style letting you shape **FrogPilot**'s behavior to match your own driving preferences! Profiles can be switched instantly using the following distance button on the steering wheel, while **"Traffic Mode"** can be enabled by simply holding down the following distance button.
---
### 📏 Speed Limit Controller (SLC)
With **"Speed Limit Controller"**, **FrogPilot** automatically adapts to the road's posted speed using information from downloaded **["OpenStreetMap"](https://www.openstreetmap.org)** maps, online **["Mapbox"](https://www.mapbox.com)** data, and the vehicle's dashboard (if supported).
Offsets let you fine-tune how closely **FrogPilot** follows posted limits across different speed ranges allowing you to cruise slightly above or below for a more natural driving experience. If no speed limit is available, you can choose whether **FrogPilot** drives at the set speed, falls back to the last known speed limit, or uses **"Experimental Mode"** to estimate one with the driving model.
Maps can be downloaded directly in settings and updated automatically on a schedule ensuring your device always has the latest speed limits!
**Note: Speed limits are only as accurate as the available speed limit data. Always stay attentive and adjust your speed when necessary!**
---
### 🎨 Themes
With **"Themes"**, you can personalize **FrogPilot**'s driving screen to make it uniquely yours! Choose from:
- **Color Schemes**
- **Icon Packs**
- **Sound Packs**
- **Turn Signal Animations**
- **Steering Wheel Icons**
Enjoy pre-existing **FrogPilot** and seasonal holiday themes, or you can create your own with the **"Theme Maker"** and even share them with the community! For extra fun, enable features like the Mario Kartstyle **"Rainbow Path"** or **"Random Events"** that add playful visual effects while you drive!
---
And lots more! From safety enhancements to personalization options, **FrogPilot** continues to evolve with features that put you in control. Check it out today for yourself!
---
🔧 Branches
------
| Branch | Install&nbsp;URL | Description | Recommended&nbsp;For |
|----------------------------|---------------------------|--------------------------------------------------------|--------------------------|
| FrogPilot | frogpilot.download | The main release branch. | Everyone |
| FrogPilot&#8209;Staging | staging.frogpilot.download| Beta branch with upcoming features. Expect bugs! | Early&nbsp;Adopters |
| FrogPilot&#8209;Testing | testing.frogpilot.download| Alpha branch with bleeding-edge features. Breaks often!| Advanced&nbsp;Testers |
| FrogPilot&#8209;Development| No :) | Active development branch. Don't use! | **FrogPilot**&nbsp;Developers|
| MAKE&#8209;PRS&#8209;HERE | No :) | Workspace for pull requests. Don't use! | Contributors |
🧰 How to Install
------
The easiest way to install **FrogPilot** is by entering this URL on the installation screen:
```
frogpilot.download
```
**DO NOT** install the **FrogPilot-Development** branch. I'm constantly breaking things on there, so unless you don't want to use **FrogPilot**, **NEVER** install it!
![](https://i.imgur.com/LTCqRqB.png)
🐞 Bug Reports / Feature Requests
------
If you run into bugs, issues, or have ideas for new features, please post about it on the **[FrogPilot Discord](https://discord.gg/frogpilot)**! Feedback helps improve **FrogPilot** and create a better experience for everyone!
To report a bug, please post it in [**#bug-reports**](https://discord.com/channels/1137853399715549214/1162100167110053888).
To request a feature, please post it in [**#feature-requests**](https://discord.com/channels/1137853399715549214/1160318669839147259).
Please include as much detail as possible! Photos, videos, log files, or anything that can help explain the issue or idea are very helpful!
I'll do my best to respond promptly, but not every request can be addressed right away. Your feedback is always appreciated and helps make **FrogPilot** the best it can be!
📋 Credits
------
* [Aidenir](https://github.com/Aidenir)
* [AlexandreSato](https://github.com/AlexandreSato)
* [cfranyota](https://github.com/cfranyota)
* [cydia2020](https://github.com/cydia2020)
* [dragonpilot-community](https://github.com/dragonpilot-community)
* [ErichMoraga](https://github.com/ErichMoraga)
* [garrettpall](https://github.com/garrettpall)
* [jakethesnake420](https://github.com/jakethesnake420)
* [jyoung8607](https://github.com/jyoung8607)
* [mike8643](https://github.com/mike8643)
* [neokii](https://github.com/neokii)
* [OPGM](https://github.com/opgm)
* [OPKR](https://github.com/openpilotkr)
* [pfeiferj](https://github.com/pfeiferj)
* [realfast](https://github.com/realfast)
* [syncword](https://github.com/syncword)
* [twilsonco](https://github.com/twilsonco)
Star History
------
[![Star History Chart](https://api.star-history.com/svg?repos=FrogAi/FrogPilot&type=Date)](https://www.star-history.com/#FrogAi/FrogPilot&Date)

View File

@ -1,74 +0,0 @@
![](dragonpilot/selfdrive/assets/dragonpilot.png)
[Read this in Chinese](README.md)
# **🐲 dragonpilot - Bringing the Spirit of the Dragon to Your Car**
**Join us on a smarter, more thoughtful driving journey.**
## **👋 Welcome, friend!**
`dragonpilot` was launched in 2019 by three early openpilot enthusiasts from the Chinese community. Our mission was simple: create a friendly space for users to share experiences, provide easier setup help, and add features tailored for local needs.
Localization has always been at the heart of what we do—starting with a fully Chinese interface. This made `dragonpilot` quickly popular in Chinese-speaking regions and helped our user base grow into one of the largest worldwide. That community support is what keeps us moving forward.
Built on top of the powerful [openpilot](https://github.com/commaai/openpilot)—an open-source driver assistance system rated by Consumer Reports as outperforming commercial offerings—we add localized refinements and user-focused features to create a driving companion that truly fits your needs. (You can also see the [original openpilot README](README_OPENPILOT.md) preserved in our repo.)
The name `dragonpilot` reflects our vision: like the dragon of mythology, it is strong and wise, guarding your safety on the road. In Chinese culture, the dragon is also a symbol of luck and strength, representing our roots and pride.
## **✨ Milestones**
Beyond carrying forward openpilot's core strengths, we've reached several milestones inspired by community feedback:
* **🚘 Always Lane Keep Assist (ALKA)**
More than a feature—it's part of the `dragonpilot` philosophy. Introduced as early as [version 0.6.2](https://github.com/dragonpilot-community/dragonpilot/blob/2861467183d62151024320447ba04d18fc3fe1e6/selfdrive/car/toyota/carstate.py#L199), first tested on a 2017 Lexus IS300h, then expanded to Toyota's lineup and beyond. ALKA helps keep your vehicle steadily centered, giving you extra confidence on the road.
* **🌐 First to add multilingual support**
Before openpilot officially supported it, we had already introduced multiple languages. `dragonpilot` fully supports Traditional Chinese, Simplified Chinese, and English.
* **💻 Only community fork to support multiple hardware platforms at once**
We uniquely worked to make the project run on EON, comma two, comma 3, and Jetson—serving the widest range of users possible.
Additionally, after the comma.ai team deprecated the comma 3 in version 0.10.0, we remain the only community fork to offer full, simultaneous support for the comma 3, comma 3X, and the O3, O3L, and O3XL (the O3 series being third-party hardware).
* **📜 Once recognized as the #1 openpilot fork**
Thanks to an active community and continuous innovation, `dragonpilot` was once the largest openpilot fork officially recognized by comma ai. This honor belongs to everyone who contributed.
## **🧑‍💻 Design Philosophy - Less is More**
As openpilot's AI grows stronger, many features that once required manual tuning are now handled by advanced models. That's why our focus has returned to **“minimal changes.”**
We aim to give you the purest, most official-like openpilot driving experience—while preserving `dragonpilot`'s classic, community-loved features. With a solid AI foundation, simplicity is strength.
## **🛠️ Hardware Journey**
From the early **EON**, to official devices like **comma two / three (C2/C3/C3X)**, to creative community builds (**C1.5, O2, O3, O3L, O3XL, etc.**), and even experiments with [**Jetson Xavier NX**](https://github.com/eFiniLan/xnxpilot).
Currently, the latest versions support: **comma3 / 3X** and community hardware like **O3 / O3L / O3XL**.
Older devices such as **EON / C1.5 / C2** are supported in the [d2 branch](https://github.com/dragonpilot-community/dragonpilot/tree/d2).
Whatever device you're on, it represents your passion for open-source driver assistance.
## **🫂 Join Us Become a “Dragon Seeker”**
`dragonpilot` thrives thanks to every user's contributions and feedback. We're an open, transparent, and welcoming community where enthusiasts can share experiences with openpilot and `dragonpilot`.
[**Join our Facebook group here!**](https://www.facebook.com/groups/930190251238639)
## **❤️ Special Thanks**
Since day one, `dragonpilot` has never asked for funding through Patreon or similar platforms. Our vision is a community where everyone learns and grows together. It's about fun, not money.
That said, we're deeply grateful to those who voluntarily supported the project. Your encouragement keeps us motivated to keep building.
[**See our sponsors**](SPONSORS.md)
### **Safety Notice**
`dragonpilot` is a driver **assistance** system, not full self-driving. It reduces fatigue and improves safety, but you must remain alert and ready to take control at all times. Always follow your local traffic laws.
**Thanks again for being here.**
**We look forward to riding the “dragon” with you on the road to smarter driving!**

View File

@ -1,107 +0,0 @@
<div align="center" style="text-align: center;">
<h1>openpilot</h1>
<p>
<b>openpilot is an operating system for robotics.</b>
<br>
Currently, it upgrades the driver assistance system in 300+ supported cars.
</p>
<h3>
<a href="https://docs.comma.ai">Docs</a>
<span> · </span>
<a href="https://docs.comma.ai/contributing/roadmap/">Roadmap</a>
<span> · </span>
<a href="https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md">Contribute</a>
<span> · </span>
<a href="https://discord.comma.ai">Community</a>
<span> · </span>
<a href="https://comma.ai/shop">Try it on a comma 3X</a>
</h3>
Quick start: `bash <(curl -fsSL openpilot.comma.ai)`
[![openpilot tests](https://github.com/commaai/openpilot/actions/workflows/selfdrive_tests.yaml/badge.svg)](https://github.com/commaai/openpilot/actions/workflows/selfdrive_tests.yaml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![X Follow](https://img.shields.io/twitter/follow/comma_ai)](https://x.com/comma_ai)
[![Discord](https://img.shields.io/discord/469524606043160576)](https://discord.comma.ai)
</div>
<table>
<tr>
<td><a href="https://youtu.be/NmBfgOanCyk" title="Video By Greer Viau"><img src="https://github.com/commaai/openpilot/assets/8762862/2f7112ae-f748-4f39-b617-fabd689c3772"></a></td>
<td><a href="https://youtu.be/VHKyqZ7t8Gw" title="Video By Logan LeGrand"><img src="https://github.com/commaai/openpilot/assets/8762862/92351544-2833-40d7-9e0b-7ef7ae37ec4c"></a></td>
<td><a href="https://youtu.be/SUIZYzxtMQs" title="A drive to Taco Bell"><img src="https://github.com/commaai/openpilot/assets/8762862/05ceefc5-2628-439c-a9b2-89ce77dc6f63"></a></td>
</tr>
</table>
Using openpilot in a car
------
To use openpilot in a car, you need four things:
1. **Supported Device:** a comma 3X, available at [comma.ai/shop](https://comma.ai/shop/comma-3x).
2. **Software:** The setup procedure for the comma 3X allows users to enter a URL for custom software. Use the URL `openpilot.comma.ai` to install the release version.
3. **Supported Car:** Ensure that you have one of [the 275+ supported cars](docs/CARS.md).
4. **Car Harness:** You will also need a [car harness](https://comma.ai/shop/car-harness) to connect your comma 3X to your car.
We have detailed instructions for [how to install the harness and device in a car](https://comma.ai/setup). Note that it's possible to run openpilot on [other hardware](https://blog.comma.ai/self-driving-car-for-free/), although it's not plug-and-play.
### Branches
| branch | URL | description |
|------------------|----------------------------------------|-------------------------------------------------------------------------------------|
| `release3` | openpilot.comma.ai | This is openpilot's release branch. |
| `release3-staging` | openpilot-test.comma.ai | This is the staging branch for releases. Use it to get new releases slightly early. |
| `nightly` | openpilot-nightly.comma.ai | This is the bleeding edge development branch. Do not expect this to be stable. |
| `nightly-dev` | installer.comma.ai/commaai/nightly-dev | Same as nightly, but includes experimental development features for some cars. |
To start developing openpilot
------
openpilot is developed by [comma](https://comma.ai/) and by users like you. We welcome both pull requests and issues on [GitHub](http://github.com/commaai/openpilot).
* Join the [community Discord](https://discord.comma.ai)
* Check out [the contributing docs](docs/CONTRIBUTING.md)
* Check out the [openpilot tools](tools/)
* Code documentation lives at https://docs.comma.ai
* Information about running openpilot lives on the [community wiki](https://github.com/commaai/openpilot/wiki)
Want to get paid to work on openpilot? [comma is hiring](https://comma.ai/jobs#open-positions) and offers lots of [bounties](https://comma.ai/bounties) for external contributors.
Safety and Testing
----
* openpilot observes [ISO26262](https://en.wikipedia.org/wiki/ISO_26262) guidelines, see [SAFETY.md](docs/SAFETY.md) for more details.
* openpilot has software-in-the-loop [tests](.github/workflows/selfdrive_tests.yaml) that run on every commit.
* The code enforcing the safety model lives in panda and is written in C, see [code rigor](https://github.com/commaai/panda#code-rigor) for more details.
* panda has software-in-the-loop [safety tests](https://github.com/commaai/panda/tree/master/tests/safety).
* Internally, we have a hardware-in-the-loop Jenkins test suite that builds and unit tests the various processes.
* panda has additional hardware-in-the-loop [tests](https://github.com/commaai/panda/blob/master/Jenkinsfile).
* We run the latest openpilot in a testing closet containing 10 comma devices continuously replaying routes.
<details>
<summary>MIT Licensed</summary>
openpilot is released under the MIT license. Some parts of the software are released under other licenses as specified.
Any user of this software shall indemnify and hold harmless Comma.ai, Inc. and its directors, officers, employees, agents, stockholders, affiliates, subcontractors and customers from and against all allegations, claims, actions, suits, demands, damages, liabilities, obligations, losses, settlements, judgments, costs and expenses (including without limitation attorneys fees and costs) which arise out of, relate to or result from any use of this software by user.
**THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT.
YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS.
NO WARRANTY EXPRESSED OR IMPLIED.**
</details>
<details>
<summary>User Data and comma Account</summary>
By default, openpilot uploads the driving data to our servers. You can also access your data through [comma connect](https://connect.comma.ai/). We use your data to train better models and improve openpilot for everyone.
openpilot is open source software: the user is free to disable data collection if they wish to do so.
openpilot logs the road-facing cameras, CAN, GPS, IMU, magnetometer, thermal sensors, crashes, and operating system logs.
The driver-facing camera and microphone are only logged if you explicitly opt-in in settings.
By using openpilot, you agree to [our Privacy Policy](https://comma.ai/privacy). You understand that use of this software or its related services will generate certain types of user data, which may be logged and stored at the sole discretion of comma. By accepting this agreement, you grant an irrevocable, perpetual, worldwide right to comma for the use of this data.
</details>

View File

@ -1,61 +1,3 @@
Version 0.10.1 (2025-09-08)
========================
* New driving model #36276
* World Model: removed global localization inputs
* World Model: 2x the number of parameters
* World Model: trained on 4x the number of segments
* VAE Compression Model: new architecture and training objective
* Driving Vision Model: trained on 4x the number of segments
* New Driver Monitoring model #36198
* Acura TLX 2021 support thanks to MVL!
* Honda City 2023 support thanks to vanillagorillaa and drFritz!
* Honda N-Box 2018 support thanks to miettal!
* Honda Odyssey 2021-25 support thanks to csouers and MVL!
* Honda Passport 2026 support thanks to vanillagorillaa and MVL!
Version 0.10.0 (2025-08-05)
========================
* New driving model
* New training architecture
* Described in our CVPR paper: "Learning to Drive from a World Model"
* Longitudinal MPC replaced by E2E planning from World Model in Experimental Mode
* Action from lateral MPC as training objective replaced by E2E planning from World Model
* Low-speed lead car ground-truth fixes
* Enable live-learned steering actuation delay
* Opt-in audio recording for dashcam video
* Acura MDX 2025 support thanks to vanillagorillaa and MVL!
* Honda Accord 2023-25 support thanks to vanillagorillaa and MVL!
* Honda CR-V 2023-25 support thanks to vanillagorillaa and MVL!
* Honda Pilot 2023-25 support thanks to vanillagorillaa and MVL!
Version 0.9.9 (2025-05-23)
========================
* New driving model
* New training architecture using parts from MLSIM
* Steering actuation delay is now learned online
* Ford Escape 2023-24 support thanks to incognitojam!
* Ford Kuga 2024 support thanks to incognitojam!
* Hyundai Nexo 2021 support thanks to sunnyhaibin!
* Tesla Model 3 and Y support thanks to lukasloetkolben!
* Lexus RC 2023 support thanks to nelsonjchen!
Version 0.9.8 (2025-02-28)
========================
* New driving model
* Model now gates applying positive acceleration in Chill mode
* New driver monitoring model
* Reduced false positives related to passengers
* Image processing pipeline moved to the ISP
* More GPU time for bigger driving models
* Power draw reduced 0.5W, which means your device runs cooler
* Added toggle to enable driver monitoring even when openpilot is not engaged
* Localizer rewritten to remove GPS dependency at runtime
* Firehose Mode for maximizing your training data uploads
* Enable openpilot longitudinal control for Ford Q3 vehicles
* New Toyota TSS2 longitudinal tune
* Rivian R1S and R1T support thanks to lukasloetkolben!
* Ford F-150, F-150 Hybrid, Mach-E, and Ranger support
Version 0.9.7 (2024-06-13)
========================
* New driving model

View File

@ -1,224 +0,0 @@
import os
import subprocess
import sys
import sysconfig
import platform
import shlex
import numpy as np
import SCons.Errors
SCons.Warnings.warningAsException(True)
Decider('MD5-timestamp')
SetOption('num_jobs', max(1, int(os.cpu_count()/2)))
AddOption('--kaitai', action='store_true', help='Regenerate kaitai struct parsers')
AddOption('--asan', action='store_true', help='turn on ASAN')
AddOption('--ubsan', action='store_true', help='turn on UBSan')
AddOption('--mutation', action='store_true', help='generate mutation-ready code')
AddOption('--ccflags', action='store', type='string', default='', help='pass arbitrary flags over the command line')
AddOption('--minimal',
action='store_false',
dest='extras',
default=os.path.exists(File('#.lfsconfig').abspath), # minimal by default on release branch (where there's no LFS)
help='the minimum build to run openpilot. no tests, tools, etc.')
# Detect platform
arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
if platform.system() == "Darwin":
arch = "Darwin"
brew_prefix = subprocess.check_output(['brew', '--prefix'], encoding='utf8').strip()
elif arch == "aarch64" and os.path.isfile('/TICI'):
arch = "larch64"
assert arch in [
"larch64", # linux tici arm64
"aarch64", # linux pc arm64
"x86_64", # linux pc x64
"Darwin", # macOS arm64 (x86 not supported)
]
env = Environment(
ENV={
"PATH": os.environ['PATH'],
"PYTHONPATH": Dir("#").abspath + ':' + Dir(f"#third_party/acados").abspath,
"ACADOS_SOURCE_DIR": Dir("#third_party/acados").abspath,
"ACADOS_PYTHON_INTERFACE_PATH": Dir("#third_party/acados/acados_template").abspath,
"TERA_PATH": Dir("#").abspath + f"/third_party/acados/{arch}/t_renderer"
},
CC='clang',
CXX='clang++',
CCFLAGS=[
"-g",
"-fPIC",
"-O2",
"-Wunused",
"-Werror",
"-Wshadow",
"-Wno-unknown-warning-option",
"-Wno-inconsistent-missing-override",
"-Wno-c99-designator",
"-Wno-reorder-init-list",
"-Wno-vla-cxx-extension",
],
CFLAGS=["-std=gnu11"],
CXXFLAGS=["-std=c++1z"],
CPPPATH=[
"#",
"#msgq",
"#third_party",
"#third_party/json11",
"#third_party/linux/include",
"#third_party/acados/include",
"#third_party/acados/include/blasfeo/include",
"#third_party/acados/include/hpipm/include",
"#third_party/catch2/include",
"#third_party/libyuv/include",
],
LIBPATH=[
"#common",
"#msgq_repo",
"#third_party",
"#selfdrive/pandad",
"#rednose/helpers",
f"#third_party/libyuv/{arch}/lib",
f"#third_party/acados/{arch}/lib",
],
RPATH=[],
CYTHONCFILESUFFIX=".cpp",
COMPILATIONDB_USE_ABSPATH=True,
REDNOSE_ROOT="#",
tools=["default", "cython", "compilation_db", "rednose_filter"],
toolpath=["#site_scons/site_tools", "#rednose_repo/site_scons/site_tools"],
)
# Arch-specific flags and paths
if arch == "larch64":
env.Append(CPPPATH=["#third_party/opencl/include"])
env.Append(LIBPATH=[
"/usr/local/lib",
"/system/vendor/lib64",
"/usr/lib/aarch64-linux-gnu",
])
arch_flags = ["-D__TICI__", "-mcpu=cortex-a57"]
env.Append(CCFLAGS=arch_flags)
env.Append(CXXFLAGS=arch_flags)
elif arch == "Darwin":
env.Append(LIBPATH=[
f"{brew_prefix}/lib",
f"{brew_prefix}/opt/openssl@3.0/lib",
f"{brew_prefix}/opt/llvm/lib/c++",
"/System/Library/Frameworks/OpenGL.framework/Libraries",
])
env.Append(CCFLAGS=["-DGL_SILENCE_DEPRECATION"])
env.Append(CXXFLAGS=["-DGL_SILENCE_DEPRECATION"])
env.Append(CPPPATH=[
f"{brew_prefix}/include",
f"{brew_prefix}/opt/openssl@3.0/include",
])
else:
env.Append(LIBPATH=[
"/usr/lib",
"/usr/local/lib",
])
# Sanitizers and extra CCFLAGS from CLI
if GetOption('asan'):
env.Append(CCFLAGS=["-fsanitize=address", "-fno-omit-frame-pointer"])
env.Append(LINKFLAGS=["-fsanitize=address"])
elif GetOption('ubsan'):
env.Append(CCFLAGS=["-fsanitize=undefined"])
env.Append(LINKFLAGS=["-fsanitize=undefined"])
_extra_cc = shlex.split(GetOption('ccflags') or '')
if _extra_cc:
env.Append(CCFLAGS=_extra_cc)
# no --as-needed on mac linker
if arch != "Darwin":
env.Append(LINKFLAGS=["-Wl,--as-needed", "-Wl,--no-undefined"])
# progress output
node_interval = 5
node_count = 0
def progress_function(node):
global node_count
node_count += node_interval
sys.stderr.write("progress: %d\n" % node_count)
if os.environ.get('SCONS_PROGRESS'):
Progress(progress_function, interval=node_interval)
# ********** Cython build environment **********
py_include = sysconfig.get_paths()['include']
envCython = env.Clone()
envCython["CPPPATH"] += [py_include, np.get_include()]
envCython["CCFLAGS"] += ["-Wno-#warnings", "-Wno-shadow", "-Wno-deprecated-declarations"]
envCython["CCFLAGS"].remove("-Werror")
envCython["LIBS"] = []
if arch == "Darwin":
envCython["LINKFLAGS"] = env["LINKFLAGS"] + ["-bundle", "-undefined", "dynamic_lookup"]
else:
envCython["LINKFLAGS"] = ["-pthread", "-shared"]
np_version = SCons.Script.Value(np.__version__)
Export('envCython', 'np_version')
Export('env', 'arch')
# Setup cache dir
cache_dir = '/data/scons_cache' if arch == "larch64" else '/tmp/scons_cache'
CacheDir(cache_dir)
Clean(["."], cache_dir)
# ********** start building stuff **********
# Build common module
SConscript(['common/SConscript'])
Import('_common')
common = [_common, 'json11', 'zmq']
Export('common')
# Build messaging (cereal + msgq + socketmaster + their dependencies)
# Enable swaglog include in submodules
env_swaglog = env.Clone()
env_swaglog['CXXFLAGS'].append('-DSWAGLOG="\\"common/swaglog.h\\""')
SConscript(['msgq_repo/SConscript'], exports={'env': env_swaglog})
SConscript(['opendbc_repo/SConscript'], exports={'env': env_swaglog})
SConscript(['cereal/SConscript'])
Import('socketmaster', 'msgq')
messaging = [socketmaster, msgq, 'capnp', 'kj',]
Export('messaging')
# Build other submodules
SConscript(['panda/SConscript'])
SConscript(['panda_tici/SConscript'])
# Build rednose library
SConscript(['rednose/SConscript'])
# Build system services
SConscript([
'system/ubloxd/SConscript',
'system/loggerd/SConscript',
])
if arch == "larch64":
SConscript(['system/camerad/SConscript'])
# Build openpilot
SConscript(['third_party/SConscript'])
SConscript(['selfdrive/SConscript'])
if Dir('#tools/cabana/').exists() and GetOption('extras'):
SConscript(['tools/replay/SConscript'])
if arch != "larch64":
SConscript(['tools/cabana/SConscript'])
env.CompilationDatabase('compile_commands.json')

View File

@ -1,26 +0,0 @@
# Sponsors 贊助者
我們誠摯感謝以下贊助者提供的硬體資源,讓專案能夠在多種平台上進行測試與驗證。
We sincerely thank the following sponsors for providing hardware resources, which enable the project to be tested and validated across multiple platforms.
---
## 贊助者列表 Sponsors
| 贊助者 Sponsor | 設備 Deices | 備註 Notes |
| -------------- | ----------- | ---------- |
| BlueGood | <ul><li>Radar Filter x 2</li><li>sDSU x1</li></ul> | - |
| Chia Chun Lee | <ul><li>C3 Quick Mount x1</li></ul> | - |
| CloudJ | <ul><li>C3X x1</li></ul> | - |
| FareWay | <ul><li>EON Quick Mount x1</li><li>C2/C3 Quick Mount x1</li></ul> | Special thanks for fixing my good old EON |
| Fred Wang | <ul><li>Oneplus 3t x1</li></ul> | - |
| Saber Huang | <ul><li>O3L x1</li></ul> | - |
| [馬威 Mr. One](https://shop61532546.taobao.com/) | <ul><li>O3 x 1</li><li>O3 (Dev) x1</li><li>O3XL x1</li><li>Red Panda x1</li><li>Panda Jungle v1 x1</li></ul> | - |
| 門文梁 | <ul><li>C1.5 x1</li></ul> | - |
---
🙏 沒有你們的支持,我們無法讓專案在這麼多硬體平台上持續成長與驗證。
Without your support, this project could not continue to grow and be validated across so many hardware platforms.

View File

@ -1,6 +1,11 @@
# What is cereal?
# What is cereal? [![cereal tests](https://github.com/commaai/cereal/workflows/tests/badge.svg?event=push)](https://github.com/commaai/cereal/actions) [![codecov](https://codecov.io/gh/commaai/cereal/branch/master/graph/badge.svg)](https://codecov.io/gh/commaai/cereal)
cereal is the messaging system for openpilot. It uses [msgq](https://github.com/commaai/msgq) as a pub/sub backend, and [Cap'n proto](https://capnproto.org/capnp-tool.html) for serialization of the structs.
cereal is both a messaging spec for robotics systems as well as generic high performance IPC pub sub messaging with a single publisher and multiple subscribers.
Imagine this use case:
* A sensor process reads gyro measurements directly from an IMU and publishes a `sensorEvents` packet
* A calibration process subscribes to the `sensorEvents` packet to use the IMU
* A localization process subscribes to the `sensorEvents` packet to use the IMU also
## Messaging Spec
@ -27,51 +32,11 @@ Forks of [openpilot](https://github.com/commaai/openpilot) might want to add thi
spec, however this could conflict with future changes made in mainline cereal/openpilot. Rebasing against mainline openpilot
then means breaking backwards-compatibility with all old logs of your fork. So we added reserved events in
[custom.capnp](custom.capnp) that we will leave empty in mainline cereal/openpilot. **If you only modify those, you can ensure your
fork will remain backwards-compatible with all versions of mainline openpilot and your fork.**
fork will remain backwards-compatible with all versions of mainline cereal/openpilot and your fork.**
An example of compatible changes:
```diff
diff --git a/cereal/custom.capnp b/cereal/custom.capnp
index 3348e859e..3365c7b98 100644
--- a/cereal/custom.capnp
+++ b/cereal/custom.capnp
@@ -10,7 +10,11 @@ $Cxx.namespace("cereal");
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
## Pub Sub Backends
-struct CustomReserved0 @0x81c2f05a394cf4af {
+struct SteeringInfo @0x81c2f05a394cf4af {
+ active @0 :Bool;
+ steeringAngleDeg @1 :Float32;
+ steeringRateDeg @2 :Float32;
+ steeringAccelDeg @3 :Float32;
}
struct CustomReserved1 @0xaedffd8f31e7b55d {
diff --git a/cereal/log.capnp b/cereal/log.capnp
index 1209f3fd9..b189f58b6 100644
--- a/cereal/log.capnp
+++ b/cereal/log.capnp
@@ -2558,14 +2558,14 @@ struct Event {
# DO change the name of the field
# DON'T change anything after the "@"
- customReservedRawData0 @124 :Data;
+ rawCanData @124 :Data;
customReservedRawData1 @125 :Data;
customReservedRawData2 @126 :Data;
# DO change the name of the field and struct
# DON'T change the ID (e.g. @107)
# DON'T change which struct it points to
- customReserved0 @107 :Custom.CustomReserved0;
+ steeringInfo @107 :Custom.SteeringInfo;
customReserved1 @108 :Custom.CustomReserved1;
customReserved2 @109 :Custom.CustomReserved2;
customReserved3 @110 :Custom.CustomReserved3;
```
---
cereal supports two backends, one based on [zmq](https://zeromq.org/) and another called [msgq](messaging/msgq.cc), a custom pub sub based on shared memory that doesn't require the bytes to pass through the kernel.
Example
---

View File

@ -1,20 +0,0 @@
Import('env', 'common', 'msgq')
cereal_dir = Dir('.')
gen_dir = Dir('gen')
# Build cereal
schema_files = ['log.capnp', 'car.capnp', 'legacy.capnp', 'custom.capnp']
env.Command([f'gen/cpp/{s}.c++' for s in schema_files] + [f'gen/cpp/{s}.h' for s in schema_files],
schema_files,
f"capnpc --src-prefix={cereal_dir.path} $SOURCES -o c++:{gen_dir.path}/cpp/")
cereal = env.Library('cereal', [f'gen/cpp/{s}.c++' for s in schema_files])
# Build messaging
services_h = env.Command(['services.h'], ['services.py'], 'python3 ' + cereal_dir.path + '/services.py > $TARGET')
env.Program('messaging/bridge', ['messaging/bridge.cc', 'messaging/msgq_to_zmq.cc'], LIBS=[msgq, common, 'pthread'])
socketmaster = env.Library('socketmaster', ['messaging/socketmaster.cc'])
Export('cereal', 'socketmaster')

View File

@ -1,11 +1,9 @@
import os
import capnp
from importlib.resources import as_file, files
CEREAL_PATH = os.path.dirname(os.path.abspath(__file__))
capnp.remove_import_hook()
with as_file(files("cereal")) as fspath:
CEREAL_PATH = fspath.as_posix()
log = capnp.load(os.path.join(CEREAL_PATH, "log.capnp"))
car = capnp.load(os.path.join(CEREAL_PATH, "car.capnp"))
custom = capnp.load(os.path.join(CEREAL_PATH, "custom.capnp"))

View File

@ -1 +0,0 @@
../opendbc_repo/opendbc/car/car.capnp

711
cereal/car.capnp Normal file
View File

@ -0,0 +1,711 @@
using Cxx = import "./include/c++.capnp";
$Cxx.namespace("cereal");
@0x8e2af1e708af8b8d;
# ******* events causing controls state machine transition *******
struct CarEvent @0x9b1657f34caf3ad3 {
name @0 :EventName;
# event types
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool; # alerts presented only when enabled or soft disabling
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool; # alerts presented regardless of openpilot state
overrideLateral @10 :Bool;
overrideLongitudinal @9 :Bool;
enum EventName @0xbaa8c5d505f727de {
canError @0;
steerUnavailable @1;
wrongGear @4;
doorOpen @5;
seatbeltNotLatched @6;
espDisabled @7;
wrongCarMode @8;
steerTempUnavailable @9;
reverseGear @10;
buttonCancel @11;
buttonEnable @12;
pedalPressed @13; # exits active state
preEnableStandstill @73; # added during pre-enable state with brake
gasPressedOverride @108; # added when user is pressing gas with no disengage on gas
steerOverride @114;
cruiseDisabled @14;
speedTooLow @17;
outOfSpace @18;
overheat @19;
calibrationIncomplete @20;
calibrationInvalid @21;
calibrationRecalibrating @117;
controlsMismatch @22;
pcmEnable @23;
pcmDisable @24;
radarFault @26;
brakeHold @28;
parkBrake @29;
manualRestart @30;
lowSpeedLockout @31;
joystickDebug @34;
steerTempUnavailableSilent @35;
resumeRequired @36;
preDriverDistracted @37;
promptDriverDistracted @38;
driverDistracted @39;
preDriverUnresponsive @43;
promptDriverUnresponsive @44;
driverUnresponsive @45;
belowSteerSpeed @46;
lowBattery @48;
accFaulted @51;
sensorDataInvalid @52;
commIssue @53;
commIssueAvgFreq @109;
tooDistracted @54;
posenetInvalid @55;
soundsUnavailable @56;
preLaneChangeLeft @57;
preLaneChangeRight @58;
laneChange @59;
lowMemory @63;
stockAeb @64;
ldw @65;
carUnrecognized @66;
invalidLkasSetting @69;
speedTooHigh @70;
laneChangeBlocked @71;
relayMalfunction @72;
stockFcw @74;
startup @75;
startupNoCar @76;
startupNoControl @77;
startupNoSecOcKey @121;
startupMaster @78;
startupNoFw @104;
fcw @79;
steerSaturated @80;
belowEngageSpeed @84;
noGps @85;
wrongCruiseMode @87;
modeldLagging @89;
deviceFalling @90;
fanMalfunction @91;
cameraMalfunction @92;
cameraFrameRate @110;
processNotRunning @95;
dashcamMode @96;
controlsInitializing @98;
usbError @99;
roadCameraError @100;
driverCameraError @101;
wideRoadCameraError @102;
highCpuUsage @105;
cruiseMismatch @106;
lkasDisabled @107;
canBusMissing @111;
controlsdLagging @112;
resumeBlocked @113;
steerTimeLimit @115;
vehicleSensorsInvalid @116;
locationdTemporaryError @103;
locationdPermanentError @118;
paramsdTemporaryError @50;
paramsdPermanentError @119;
actuatorsApiUnavailable @120;
radarCanErrorDEPRECATED @15;
communityFeatureDisallowedDEPRECATED @62;
radarCommIssueDEPRECATED @67;
driverMonitorLowAccDEPRECATED @68;
gasUnavailableDEPRECATED @3;
dataNeededDEPRECATED @16;
modelCommIssueDEPRECATED @27;
ipasOverrideDEPRECATED @33;
geofenceDEPRECATED @40;
driverMonitorOnDEPRECATED @41;
driverMonitorOffDEPRECATED @42;
calibrationProgressDEPRECATED @47;
invalidGiraffeHondaDEPRECATED @49;
invalidGiraffeToyotaDEPRECATED @60;
internetConnectivityNeededDEPRECATED @61;
whitePandaUnsupportedDEPRECATED @81;
commIssueWarningDEPRECATED @83;
focusRecoverActiveDEPRECATED @86;
neosUpdateRequiredDEPRECATED @88;
modelLagWarningDEPRECATED @93;
startupOneplusDEPRECATED @82;
startupFuzzyFingerprintDEPRECATED @97;
noTargetDEPRECATED @25;
brakeUnavailableDEPRECATED @2;
plannerErrorDEPRECATED @32;
gpsMalfunctionDEPRECATED @94;
}
}
# ******* main car state @ 100hz *******
# all speeds in m/s
struct CarState {
events @13 :List(CarEvent);
# CAN health
canValid @26 :Bool; # invalid counter/checksums
canTimeout @40 :Bool; # CAN bus dropped out
canErrorCounter @48 :UInt32;
# car speed
vEgo @1 :Float32; # best estimate of speed
aEgo @16 :Float32; # best estimate of acceleration
vEgoRaw @17 :Float32; # unfiltered speed from CAN sensors
vEgoCluster @44 :Float32; # best estimate of speed shown on car's instrument cluster, used for UI
yawRate @22 :Float32; # best estimate of yaw rate
standstill @18 :Bool;
wheelSpeeds @2 :WheelSpeeds;
# gas pedal, 0.0-1.0
gas @3 :Float32; # this is user pedal only
gasPressed @4 :Bool; # this is user pedal only
engineRpm @46 :Float32;
# brake pedal, 0.0-1.0
brake @5 :Float32; # this is user pedal only
brakePressed @6 :Bool; # this is user pedal only
regenBraking @45 :Bool; # this is user pedal only
parkingBrake @39 :Bool;
brakeHoldActive @38 :Bool;
# steering wheel
steeringAngleDeg @7 :Float32;
steeringAngleOffsetDeg @37 :Float32; # Offset betweens sensors in case there multiple
steeringRateDeg @15 :Float32;
steeringTorque @8 :Float32; # TODO: standardize units
steeringTorqueEps @27 :Float32; # TODO: standardize units
steeringPressed @9 :Bool; # if the user is using the steering wheel
steerFaultTemporary @35 :Bool; # temporary EPS fault
steerFaultPermanent @36 :Bool; # permanent EPS fault
stockAeb @30 :Bool;
stockFcw @31 :Bool;
espDisabled @32 :Bool;
accFaulted @42 :Bool;
carFaultedNonCritical @47 :Bool; # some ECU is faulted, but car remains controllable
# cruise state
cruiseState @10 :CruiseState;
# gear
gearShifter @14 :GearShifter;
# button presses
buttonEvents @11 :List(ButtonEvent);
leftBlinker @20 :Bool;
rightBlinker @21 :Bool;
genericToggle @23 :Bool;
# lock info
doorOpen @24 :Bool;
seatbeltUnlatched @25 :Bool;
# clutch (manual transmission only)
clutchPressed @28 :Bool;
# blindspot sensors
leftBlindspot @33 :Bool; # Is there something blocking the left lane change
rightBlindspot @34 :Bool; # Is there something blocking the right lane change
fuelGauge @41 :Float32; # battery or fuel tank level from 0.0 to 1.0
charging @43 :Bool;
# process meta
cumLagMs @50 :Float32;
struct WheelSpeeds {
# optional wheel speeds
fl @0 :Float32;
fr @1 :Float32;
rl @2 :Float32;
rr @3 :Float32;
}
struct CruiseState {
enabled @0 :Bool;
speed @1 :Float32;
speedCluster @6 :Float32; # Set speed as shown on instrument cluster
available @2 :Bool;
speedOffset @3 :Float32;
standstill @4 :Bool;
nonAdaptive @5 :Bool;
}
enum GearShifter {
unknown @0;
park @1;
drive @2;
neutral @3;
reverse @4;
sport @5;
low @6;
brake @7;
eco @8;
manumatic @9;
}
# send on change
struct ButtonEvent {
pressed @0 :Bool;
type @1 :Type;
enum Type {
unknown @0;
leftBlinker @1;
rightBlinker @2;
accelCruise @3;
decelCruise @4;
cancel @5;
altButton1 @6;
altButton2 @7;
altButton3 @8;
setCruise @9;
resumeCruise @10;
gapAdjustCruise @11;
}
}
# deprecated
errorsDEPRECATED @0 :List(CarEvent.EventName);
brakeLightsDEPRECATED @19 :Bool;
steeringRateLimitedDEPRECATED @29 :Bool;
canMonoTimesDEPRECATED @12: List(UInt64);
canRcvTimeoutDEPRECATED @49 :Bool;
}
# ******* radar state @ 20hz *******
struct RadarData @0x888ad6581cf0aacb {
errors @0 :List(Error);
points @1 :List(RadarPoint);
enum Error {
canError @0;
fault @1;
wrongConfig @2;
}
# similar to LiveTracks
# is one timestamp valid for all? I think so
struct RadarPoint {
trackId @0 :UInt64; # no trackId reuse
# these 3 are the minimum required
dRel @1 :Float32; # m from the front bumper of the car
yRel @2 :Float32; # m
vRel @3 :Float32; # m/s
# these are optional and valid if they are not NaN
aRel @4 :Float32; # m/s^2
yvRel @5 :Float32; # m/s
# some radars flag measurements VS estimates
measured @6 :Bool;
}
# deprecated
canMonoTimesDEPRECATED @2 :List(UInt64);
}
# ******* car controls @ 100hz *******
struct CarControl {
# must be true for any actuator commands to work
enabled @0 :Bool;
latActive @11: Bool;
longActive @12: Bool;
# Actuator commands as computed by controlsd
actuators @6 :Actuators;
# moved to CarOutput
actuatorsOutputDEPRECATED @10 :Actuators;
leftBlinker @15: Bool;
rightBlinker @16: Bool;
orientationNED @13 :List(Float32);
angularVelocity @14 :List(Float32);
cruiseControl @4 :CruiseControl;
hudControl @5 :HUDControl;
struct Actuators {
# range from 0.0 - 1.0
gas @0: Float32;
brake @1: Float32;
# range from -1.0 - 1.0
steer @2: Float32;
# value sent over can to the car
steerOutputCan @8: Float32;
steeringAngleDeg @3: Float32;
curvature @7: Float32;
speed @6: Float32; # m/s
accel @4: Float32; # m/s^2
longControlState @5: LongControlState;
enum LongControlState @0xe40f3a917d908282{
off @0;
pid @1;
stopping @2;
starting @3;
}
}
struct CruiseControl {
cancel @0: Bool;
resume @1: Bool;
override @4: Bool;
speedOverrideDEPRECATED @2: Float32;
accelOverrideDEPRECATED @3: Float32;
}
struct HUDControl {
speedVisible @0: Bool;
setSpeed @1: Float32;
lanesVisible @2: Bool;
leadVisible @3: Bool;
visualAlert @4: VisualAlert;
audibleAlert @5: AudibleAlert;
rightLaneVisible @6: Bool;
leftLaneVisible @7: Bool;
rightLaneDepart @8: Bool;
leftLaneDepart @9: Bool;
leadDistanceBars @10: Int8; # 1-3: 1 is closest, 3 is farthest. some ports may utilize 2-4 bars instead
enum VisualAlert {
# these are the choices from the Honda
# map as good as you can for your car
none @0;
fcw @1;
steerRequired @2;
brakePressed @3;
wrongGear @4;
seatbeltUnbuckled @5;
speedTooHigh @6;
ldw @7;
}
enum AudibleAlert {
none @0;
engage @1;
disengage @2;
refuse @3;
warningSoft @4;
warningImmediate @5;
prompt @6;
promptRepeat @7;
promptDistracted @8;
}
}
gasDEPRECATED @1 :Float32;
brakeDEPRECATED @2 :Float32;
steeringTorqueDEPRECATED @3 :Float32;
activeDEPRECATED @7 :Bool;
rollDEPRECATED @8 :Float32;
pitchDEPRECATED @9 :Float32;
}
struct CarOutput {
# Any car specific rate limits or quirks applied by
# the CarController are reflected in actuatorsOutput
# and matches what is sent to the car
actuatorsOutput @0 :CarControl.Actuators;
}
# ****** car param ******
struct CarParams {
carName @0 :Text;
carFingerprint @1 :Text;
fuzzyFingerprint @55 :Bool;
notCar @66 :Bool; # flag for non-car robotics platforms
pcmCruise @3 :Bool; # is openpilot's state tied to the PCM's cruise state?
enableDsu @5 :Bool; # driving support unit
enableBsm @56 :Bool; # blind spot monitoring
flags @64 :UInt32; # flags for car specific quirks
experimentalLongitudinalAvailable @71 :Bool;
minEnableSpeed @7 :Float32;
minSteerSpeed @8 :Float32;
safetyConfigs @62 :List(SafetyConfig);
alternativeExperience @65 :Int16; # panda flag for features like no disengage on gas
# Car docs fields
maxLateralAccel @68 :Float32;
autoResumeSng @69 :Bool; # describes whether car can resume from a stop automatically
# things about the car in the manual
mass @17 :Float32; # [kg] curb weight: all fluids no cargo
wheelbase @18 :Float32; # [m] distance from rear axle to front axle
centerToFront @19 :Float32; # [m] distance from center of mass to front axle
steerRatio @20 :Float32; # [] ratio of steering wheel angle to front wheel angle
steerRatioRear @21 :Float32; # [] ratio of steering wheel angle to rear wheel angle (usually 0)
# things we can derive
rotationalInertia @22 :Float32; # [kg*m2] body rotational inertia
tireStiffnessFactor @72 :Float32; # scaling factor used in calculating tireStiffness[Front,Rear]
tireStiffnessFront @23 :Float32; # [N/rad] front tire coeff of stiff
tireStiffnessRear @24 :Float32; # [N/rad] rear tire coeff of stiff
longitudinalTuning @25 :LongitudinalPIDTuning;
lateralParams @48 :LateralParams;
lateralTuning :union {
pid @26 :LateralPIDTuning;
indiDEPRECATED @27 :LateralINDITuning;
lqrDEPRECATED @40 :LateralLQRTuning;
torque @67 :LateralTorqueTuning;
}
steerLimitAlert @28 :Bool;
steerLimitTimer @47 :Float32; # time before steerLimitAlert is issued
vEgoStopping @29 :Float32; # Speed at which the car goes into stopping state
vEgoStarting @59 :Float32; # Speed at which the car goes into starting state
stoppingControl @31 :Bool; # Does the car allow full control even at lows speeds when stopping
steerControlType @34 :SteerControlType;
radarUnavailable @35 :Bool; # True when radar objects aren't visible on CAN or aren't parsed out
stopAccel @60 :Float32; # Required acceleration to keep vehicle stationary
stoppingDecelRate @52 :Float32; # m/s^2/s while trying to stop
startAccel @32 :Float32; # Required acceleration to get car moving
startingState @70 :Bool; # Does this car make use of special starting state
steerActuatorDelay @36 :Float32; # Steering wheel actuator delay in seconds
longitudinalActuatorDelay @58 :Float32; # Gas/Brake actuator delay in seconds
openpilotLongitudinalControl @37 :Bool; # is openpilot doing the longitudinal control?
carVin @38 :Text; # VIN number queried during fingerprinting
dashcamOnly @41: Bool;
passive @73: Bool; # is openpilot in control?
transmissionType @43 :TransmissionType;
carFw @44 :List(CarFw);
radarTimeStep @45: Float32 = 0.05; # time delta between radar updates, 20Hz is very standard
fingerprintSource @49: FingerprintSource;
networkLocation @50 :NetworkLocation; # Where Panda/C2 is integrated into the car's CAN network
wheelSpeedFactor @63 :Float32; # Multiplier on wheels speeds to computer actual speeds
secOcRequired @74 :Bool; # Car requires SecOC message authentication to operate
secOcKeyAvailable @75 :Bool; # Stored SecOC key loaded from params
struct SafetyConfig {
safetyModel @0 :SafetyModel;
safetyParam @3 :UInt16;
safetyParamDEPRECATED @1 :Int16;
safetyParam2DEPRECATED @2 :UInt32;
}
struct LateralParams {
torqueBP @0 :List(Int32);
torqueV @1 :List(Int32);
}
struct LateralPIDTuning {
kpBP @0 :List(Float32);
kpV @1 :List(Float32);
kiBP @2 :List(Float32);
kiV @3 :List(Float32);
kf @4 :Float32;
}
struct LateralTorqueTuning {
useSteeringAngle @0 :Bool;
friction @3 :Float32;
steeringAngleDeadzoneDeg @5 :Float32;
latAccelFactor @6 :Float32;
latAccelOffset @7 :Float32;
kpDEPRECATED @1 :Float32;
kiDEPRECATED @2 :Float32;
kfDEPRECATED @4 :Float32;
kdDEPRECATED @8 : Float32;
}
struct LongitudinalPIDTuning {
kpBP @0 :List(Float32);
kpV @1 :List(Float32);
kiBP @2 :List(Float32);
kiV @3 :List(Float32);
kfDEPRECATED @6 :Float32;
deadzoneBP @4 :List(Float32);
deadzoneV @5 :List(Float32);
}
struct LateralINDITuning {
outerLoopGainBP @4 :List(Float32);
outerLoopGainV @5 :List(Float32);
innerLoopGainBP @6 :List(Float32);
innerLoopGainV @7 :List(Float32);
timeConstantBP @8 :List(Float32);
timeConstantV @9 :List(Float32);
actuatorEffectivenessBP @10 :List(Float32);
actuatorEffectivenessV @11 :List(Float32);
outerLoopGainDEPRECATED @0 :Float32;
innerLoopGainDEPRECATED @1 :Float32;
timeConstantDEPRECATED @2 :Float32;
actuatorEffectivenessDEPRECATED @3 :Float32;
}
struct LateralLQRTuning {
scale @0 :Float32;
ki @1 :Float32;
dcGain @2 :Float32;
# State space system
a @3 :List(Float32);
b @4 :List(Float32);
c @5 :List(Float32);
k @6 :List(Float32); # LQR gain
l @7 :List(Float32); # Kalman gain
}
enum SafetyModel {
silent @0;
hondaNidec @1;
toyota @2;
elm327 @3;
gm @4;
hondaBoschGiraffe @5;
ford @6;
cadillac @7;
hyundai @8;
chrysler @9;
tesla @10;
subaru @11;
gmPassive @12;
mazda @13;
nissan @14;
volkswagen @15;
toyotaIpas @16;
allOutput @17;
gmAscm @18;
noOutput @19; # like silent but without silent CAN TXs
hondaBosch @20;
volkswagenPq @21;
subaruPreglobal @22; # pre-Global platform
hyundaiLegacy @23;
hyundaiCommunity @24;
volkswagenMlb @25;
hongqi @26;
body @27;
hyundaiCanfd @28;
volkswagenMqbEvo @29;
chryslerCusw @30;
psa @31;
}
enum SteerControlType {
torque @0;
angle @1;
curvatureDEPRECATED @2;
}
enum TransmissionType {
unknown @0;
automatic @1; # Traditional auto, including DSG
manual @2; # True "stick shift" only
direct @3; # Electric vehicle or other direct drive
cvt @4;
}
struct CarFw {
ecu @0 :Ecu;
fwVersion @1 :Data;
address @2 :UInt32;
subAddress @3 :UInt8;
responseAddress @4 :UInt32;
request @5 :List(Data);
brand @6 :Text;
bus @7 :UInt8;
logging @8 :Bool;
obdMultiplexing @9 :Bool;
}
enum Ecu {
eps @0;
abs @1;
fwdRadar @2;
fwdCamera @3;
engine @4;
unknown @5;
transmission @8; # Transmission Control Module
hybrid @18; # hybrid control unit, e.g. Chrysler's HCP, Honda's IMA Control Unit, Toyota's hybrid control computer
srs @9; # airbag
gateway @10; # can gateway
hud @11; # heads up display
combinationMeter @12; # instrument cluster
electricBrakeBooster @15;
shiftByWire @16;
adas @19;
cornerRadar @21;
hvac @20;
parkingAdas @7; # parking assist system ECU, e.g. Toyota's IPAS, Hyundai's RSPA, etc.
epb @22; # electronic parking brake
telematics @23;
body @24; # body control module
# Toyota only
dsu @6;
# Honda only
vsa @13; # Vehicle Stability Assist
programmedFuelInjection @14;
debug @17;
}
enum FingerprintSource {
can @0;
fw @1;
fixed @2;
}
enum NetworkLocation {
fwdCamera @0; # Standard/default integration at LKAS camera
gateway @1; # Integration at vehicle's CAN gateway
}
enableGasInterceptor @2 :Bool;
enableCameraDEPRECATED @4 :Bool;
enableApgsDEPRECATED @6 :Bool;
steerRateCostDEPRECATED @33 :Float32;
isPandaBlackDEPRECATED @39 :Bool;
hasStockCameraDEPRECATED @57 :Bool;
safetyParamDEPRECATED @10 :Int16;
safetyModelDEPRECATED @9 :SafetyModel;
safetyModelPassiveDEPRECATED @42 :SafetyModel = silent;
minSpeedCanDEPRECATED @51 :Float32;
communityFeatureDEPRECATED @46: Bool;
startingAccelRateDEPRECATED @53 :Float32;
steerMaxBPDEPRECATED @11 :List(Float32);
steerMaxVDEPRECATED @12 :List(Float32);
gasMaxBPDEPRECATED @13 :List(Float32);
gasMaxVDEPRECATED @14 :List(Float32);
brakeMaxBPDEPRECATED @15 :List(Float32);
brakeMaxVDEPRECATED @16 :List(Float32);
directAccelControlDEPRECATED @30 :Bool;
maxSteeringAngleDegDEPRECATED @54 :Float32;
longitudinalActuatorDelayLowerBoundDEPRECATEDDEPRECATED @61 :Float32;
}

View File

@ -1,74 +1,246 @@
using Cxx = import "./include/c++.capnp";
$Cxx.namespace("cereal");
using Car = import "car.capnp";
@0xb526ba661d550a59;
# custom.capnp: a home for empty structs reserved for custom forks
# These structs are guaranteed to remain reserved and empty in mainline
# cereal, so use these if you want custom events in your fork.
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
# you can rename the struct, but don't change the identifier
struct FrogPilotCarControl {
hudControl @0 :HUDControl;
struct DpControlsState @0x81c2f05a394cf4af {
alkaActive @0 :Bool;
struct HUDControl {
audibleAlert @0 :AudibleAlert;
enum AudibleAlert {
none @0;
engage @1;
disengage @2;
refuse @3;
warningSoft @4;
warningImmediate @5;
prompt @6;
promptRepeat @7;
promptDistracted @8;
# Random Events
angry @9;
continued @10;
dejaVu @11;
doc @12;
fart @13;
firefox @14;
goat @15;
hal9000 @16;
mail @17;
nessie @18;
noice @19;
startup @20;
thisIsFine @21;
uwu @22;
}
}
}
struct ModelExt @0xaedffd8f31e7b55d {
leftEdgeDetected @0 :Bool;
rightEdgeDetected @1 :Bool;
struct FrogPilotCarEvent @0x81c2f05a394cf4af {
name @0 :EventName;
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool;
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool;
overrideLateral @10 :Bool;
overrideLongitudinal @9 :Bool;
enum EventName @0xaedffd8f31e7b55d {
blockUser @0;
customStartupAlert @1;
forcingStop @2;
goatSteerSaturated @3;
greenLight @4;
holidayActive @5;
laneChangeBlockedLoud @6;
leadDeparting @7;
noLaneAvailable @8;
openpilotCrashed @9;
pedalInterceptorNoBrake @10;
speedLimitChanged @11;
torqueNNLoad @12;
trafficModeActive @13;
trafficModeInactive @14;
turningLeft @15;
turningRight @16;
# Random Events
accel30 @17;
accel35 @18;
accel40 @19;
dejaVuCurve @20;
firefoxSteerSaturated @21;
hal9000 @22;
openpilotCrashedRandomEvent @23;
thisIsFineSteerSaturated @24;
toBeContinued @25;
vCruise69 @26;
yourFrogTriedToKillMe @27;
youveGotMail @28;
}
}
struct CustomReserved2 @0xf35cc4560bbf6ec2 {
struct FrogPilotCarParams @0xf35cc4560bbf6ec2 {
canUsePedal @0 :Bool;
canUseSDSU @1 :Bool;
fpFlags @2 :UInt32;
isHDA2 @3 :Bool;
openpilotLongitudinalControlDisabled @4 :Bool;
safetyConfigs @5 :List(SafetyConfig);
struct SafetyConfig {
safetyParam @0 :UInt16;
}
}
struct CustomReserved3 @0xda96579883444c35 {
struct FrogPilotCarState @0xda96579883444c35 {
accelPressed @0 :Bool;
alwaysOnLateralAllowed @1 :Bool;
alwaysOnLateralEnabled @2 :Bool;
brakeLights @3 :Bool;
dashboardSpeedLimit @4 :Float32;
decelPressed @5 :Bool;
distancePressed @6 :Bool;
distanceLongPressed @7 :Bool;
distanceVeryLongPressed @8 :Bool;
ecoGear @9 :Bool;
forceCoast @10 :Bool;
pauseLateral @11 :Bool;
pauseLongitudinal @12 :Bool;
sportGear @13 :Bool;
trafficModeEnabled @14 :Bool;
struct ButtonEvent {
enum Type {
lkas @0;
}
}
}
struct CustomReserved4 @0x80ae746ee2596b11 {
struct FrogPilotControlsState @0x80ae746ee2596b11 {
alertStatus @0 :AlertStatus;
alertText1 @1 :Text;
alertText2 @2 :Text;
alertSize @3 :AlertSize;
alertBlinkingRate @4 :Float32;
alertType @5 :Text;
alertSound @6 :Car.CarControl.HUDControl.AudibleAlert;
enum AlertSize {
none @0; # don't display the alert
small @1; # small box
mid @2; # mid screen
full @3; # full screen
}
struct CustomReserved5 @0xa5cd762cd951a455 {
enum AlertStatus {
normal @0; # low priority alert for user's convenience
userPrompt @1; # mid priority alert that might require user intervention
critical @2; # high priority alert that needs immediate user intervention
frogpilot @3; # FrogPilot startup alert
}
}
struct CustomReserved6 @0xf98d843bfd7004a3 {
struct FrogPilotDeviceState @0xa5cd762cd951a455 {
freeSpace @0 :Int16;
usedSpace @1 :Int16;
}
struct CustomReserved7 @0xb86e6369214c01c8 {
struct FrogPilotModelDataV2 @0xf98d843bfd7004a3 {
turnDirection @0 :TurnDirection;
enum TurnDirection {
none @0;
turnLeft @1;
turnRight @2;
}
}
struct CustomReserved8 @0xf416ec09499d9d19 {
struct FrogPilotNavigation @0xf416ec09499d9d19 {
approachingIntersection @0 :Bool;
approachingTurn @1 :Bool;
navigationSpeedLimit @2 :Float32;
}
struct CustomReserved9 @0xa1680744031fdb2d {
struct FrogPilotPlan @0xa1680744031fdb2d {
accelerationJerk @0 :Float32;
accelerationJerkStock @1 :Float32;
cscControllingSpeed @2 :Bool;
cscSpeed @3 :Float32;
cscTraining @4 :Bool;
dangerJerk @5 :Float32;
desiredFollowDistance @6 :Int64;
experimentalMode @7 :Bool;
forcingStop @8 :Bool;
forcingStopLength @9 :Float32;
frogpilotEvents @10 :List(FrogPilotCarEvent);
increasedStoppedDistance @11 :Float32;
lateralCheck @12 :Bool;
laneWidthLeft @13 :Float32;
laneWidthRight @14 :Float32;
maxAcceleration @15 :Float32;
minAcceleration @16 :Float32;
redLight @17 :Bool;
roadCurvature @18 :Float32;
slcMapSpeedLimit @19 :Float32;
slcMapboxSpeedLimit @20 :Float32;
slcNextSpeedLimit @21 :Float32;
slcOverriddenSpeed @22 :Float32;
slcSpeedLimit @23 :Float32;
slcSpeedLimitOffset @24 :Float32;
slcSpeedLimitSource @25 :Text;
speedJerk @26 :Float32;
speedJerkStock @27 :Float32;
speedLimitChanged @28 :Bool;
tFollow @29 :Float32;
themeUpdated @30 :Bool;
togglesUpdated @31 :Bool;
trackingLead @32 :Bool;
unconfirmedSlcSpeedLimit @33 :Float32;
vCruise @34 :Float32;
weatherDaytime @35 :Bool;
weatherId @36 :Int16;
}
struct CustomReserved10 @0xcb9fd56c7057593a {
}
struct FrogPilotRadarState @0xcb9fd56c7057593a {
leadLeft @0 :LeadData;
leadRight @1 :LeadData;
struct CustomReserved11 @0xc2243c65e0340384 {
}
struct LeadData {
dRel @0 :Float32;
yRel @1 :Float32;
vRel @2 :Float32;
aRel @3 :Float32;
vLead @4 :Float32;
dPath @6 :Float32;
vLat @7 :Float32;
vLeadK @8 :Float32;
aLeadK @9 :Float32;
fcw @10 :Bool;
status @11 :Bool;
aLeadTau @12 :Float32;
modelProb @13 :Float32;
radar @14 :Bool;
radarTrackId @15 :Int32 = -1;
struct CustomReserved12 @0x9ccdc8676701b412 {
aLeadDEPRECATED @5 :Float32;
}
struct CustomReserved13 @0xcd96dafb67a082d0 {
}
struct CustomReserved14 @0xb057204d7deadf3f {
}
struct CustomReserved15 @0xbd443b539493bc68 {
}
struct CustomReserved16 @0xfc6241ed8877b611 {
}
struct CustomReserved17 @0xa30662f84033036c {
}
struct CustomReserved18 @0xc86a3d38d13eb3ef {
}
struct CustomReserved19 @0xa4f1eb3323f5f582 {
}

7092
cereal/gen/cpp/car.capnp.c++ Normal file

File diff suppressed because it is too large Load Diff

9135
cereal/gen/cpp/car.capnp.h Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

11415
cereal/gen/cpp/legacy.capnp.h Normal file

File diff suppressed because it is too large Load Diff

32321
cereal/gen/cpp/log.capnp.c++ Normal file

File diff suppressed because it is too large Load Diff

55703
cereal/gen/cpp/log.capnp.h Normal file

File diff suppressed because it is too large Load Diff

BIN
cereal/libcereal_shared.so Executable file

Binary file not shown.

View File

@ -17,124 +17,6 @@ struct Map(Key, Value) {
}
}
struct OnroadEvent @0xc4fa6047f024e718 {
name @0 :EventName;
# event types
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool; # alerts presented only when enabled or soft disabling
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool; # alerts presented regardless of openpilot state
overrideLateral @10 :Bool;
overrideLongitudinal @9 :Bool;
enum EventName @0x91f1992a1f77fb03 {
canError @0;
steerUnavailable @1;
wrongGear @2;
doorOpen @3;
seatbeltNotLatched @4;
espDisabled @5;
wrongCarMode @6;
steerTempUnavailable @7;
reverseGear @8;
buttonCancel @9;
buttonEnable @10;
pedalPressed @11; # exits active state
preEnableStandstill @12; # added during pre-enable state with brake
gasPressedOverride @13; # added when user is pressing gas with no disengage on gas
steerOverride @14;
steerDisengage @94; # exits active state
cruiseDisabled @15;
speedTooLow @16;
outOfSpace @17;
overheat @18;
calibrationIncomplete @19;
calibrationInvalid @20;
calibrationRecalibrating @21;
controlsMismatch @22;
pcmEnable @23;
pcmDisable @24;
radarFault @25;
radarTempUnavailable @93;
brakeHold @26;
parkBrake @27;
manualRestart @28;
joystickDebug @29;
longitudinalManeuver @30;
steerTempUnavailableSilent @31;
resumeRequired @32;
preDriverDistracted @33;
promptDriverDistracted @34;
driverDistracted @35;
preDriverUnresponsive @36;
promptDriverUnresponsive @37;
driverUnresponsive @38;
belowSteerSpeed @39;
lowBattery @40;
accFaulted @41;
sensorDataInvalid @42;
commIssue @43;
commIssueAvgFreq @44;
tooDistracted @45;
posenetInvalid @46;
preLaneChangeLeft @48;
preLaneChangeRight @49;
laneChange @50;
lowMemory @51;
stockAeb @52;
ldw @53;
carUnrecognized @54;
invalidLkasSetting @55;
speedTooHigh @56;
laneChangeBlocked @57;
relayMalfunction @58;
stockFcw @59;
startup @60;
startupNoCar @61;
startupNoControl @62;
startupNoSecOcKey @63;
startupMaster @64;
fcw @65;
steerSaturated @66;
belowEngageSpeed @67;
noGps @68;
wrongCruiseMode @69;
modeldLagging @70;
deviceFalling @71;
fanMalfunction @72;
cameraMalfunction @73;
cameraFrameRate @74;
processNotRunning @75;
dashcamMode @76;
selfdriveInitializing @77;
usbError @78;
cruiseMismatch @79;
canBusMissing @80;
selfdrivedLagging @81;
resumeBlocked @82;
steerTimeLimit @83;
vehicleSensorsInvalid @84;
locationdTemporaryError @85;
locationdPermanentError @86;
paramsdTemporaryError @87;
paramsdPermanentError @88;
actuatorsApiUnavailable @89;
espActive @90;
personalityChanged @91;
aeb @92;
userBookmark @95;
excessiveActuation @96;
audioFeedback @97;
soundsUnavailableDEPRECATED @47;
}
}
enum LongitudinalPersonality {
aggressive @0;
standard @1;
@ -156,10 +38,6 @@ struct InitData {
gitBranch @11 :Text;
gitRemote @13 :Text;
# this is source commit for prebuilt branches
gitSrcCommit @23 :Text;
gitSrcCommitDate @24 :Text;
androidProperties @16 :Map(Text, Text);
pandaInfo @8 :PandaInfo;
@ -259,6 +137,8 @@ struct FrameData {
requestId @28 :UInt32;
encodeId @1 :UInt32;
frameType @7 :FrameType;
# Timestamps
timestampEof @2 :UInt64;
timestampSof @8 :UInt64;
@ -278,7 +158,7 @@ struct FrameData {
temperaturesC @24 :List(Float32);
enum FrameTypeDEPRECATED {
enum FrameType {
unknown @0;
neo @1;
chffrAndroid @2;
@ -295,7 +175,6 @@ struct FrameData {
frameLengthDEPRECATED @3 :Int32;
globalGainDEPRECATED @5 :Int32;
frameTypeDEPRECATED @7 :FrameTypeDEPRECATED;
androidCaptureResultDEPRECATED @9 :AndroidCaptureResult;
lensPosDEPRECATED @11 :Int32;
lensSagDEPRECATED @12 :Float32;
@ -418,7 +297,6 @@ struct GpsLocationData {
speedAccuracy @12 :Float32;
hasFix @13 :Bool;
satelliteCount @14 :Int8;
enum SensorSource {
android @0;
@ -459,9 +337,9 @@ enum LaneChangeDirection {
struct CanData {
address @0 :UInt32;
busTime @1 :UInt16;
dat @2 :Data;
src @3 :UInt8;
busTimeDEPRECATED @1 :UInt16;
}
struct DeviceState @0xa4d8b5af2aa492eb {
@ -492,13 +370,10 @@ struct DeviceState @0xa4d8b5af2aa492eb {
# device thermals
cpuTempC @26 :List(Float32);
gpuTempC @27 :List(Float32);
dspTempC @49 :Float32;
memoryTempC @28 :Float32;
nvmeTempC @35 :List(Float32);
modemTempC @36 :List(Float32);
pmicTempC @39 :List(Float32);
intakeTempC @46 :Float32;
exhaustTempC @47 :Float32;
caseTempC @48 :Float32;
maxTempC @44 :Float32; # max of other temps, used to control fan
thermalZones @38 :List(ThermalZone);
thermalStatus @14 :ThermalStatus;
@ -569,7 +444,6 @@ struct DeviceState @0xa4d8b5af2aa492eb {
chargingDisabledDEPRECATED @18 :Bool;
usbOnlineDEPRECATED @12 :Bool;
ambientTempCDEPRECATED @30 :Float32;
nvmeTempCDEPRECATED @35 :List(Float32);
}
struct PandaState @0xa7649e2575e4591e {
@ -585,8 +459,9 @@ struct PandaState @0xa7649e2575e4591e {
heartbeatLost @22 :Bool;
interruptLoad @25 :Float32;
fanPower @28 :UInt8;
fanStallCount @34 :UInt8;
spiErrorCount @33 :UInt16;
spiChecksumErrorCount @33 :UInt16;
harnessStatus @21 :HarnessStatus;
sbu1Voltage @35 :Float32;
@ -713,7 +588,6 @@ struct PandaState @0xa7649e2575e4591e {
usbPowerModeDEPRECATED @12 :PeripheralState.UsbPowerModeDEPRECATED;
safetyParamDEPRECATED @20 :Int16;
safetyParam2DEPRECATED @26 :UInt32;
fanStallCountDEPRECATED @34 :UInt8;
}
struct PeripheralState {
@ -734,10 +608,11 @@ struct PeripheralState {
struct RadarState @0x9a185389d6fdd05f {
mdMonoTime @6 :UInt64;
carStateMonoTime @11 :UInt64;
radarErrors @13 :Car.RadarData.Error;
radarErrors @12 :List(Car.RadarData.Error);
leadOne @3 :LeadData;
leadTwo @4 :LeadData;
cumLagMs @5 :Float32;
struct LeadData {
dRel @0 :Float32;
@ -767,8 +642,6 @@ struct RadarState @0x9a185389d6fdd05f {
calCycleDEPRECATED @8 :Int32;
calPercDEPRECATED @9 :Int8;
canMonoTimesDEPRECATED @10 :List(UInt64);
cumLagMsDEPRECATED @5 :Float32;
radarErrorsDEPRECATED @12 :List(Car.RadarData.ErrorDEPRECATED);
}
struct LiveCalibrationData {
@ -799,7 +672,7 @@ struct LiveCalibrationData {
}
}
struct LiveTracksDEPRECATED {
struct LiveTracks {
trackId @0 :Int32;
dRel @1 :Float32;
yRel @2 :Float32;
@ -812,25 +685,53 @@ struct LiveTracksDEPRECATED {
oncoming @9 :Bool;
}
struct SelfdriveState {
# high level system state
state @0 :OpenpilotState;
enabled @1 :Bool;
active @2 :Bool;
engageable @9 :Bool; # can OP be engaged?
struct ControlsState @0x97ff69c53601abf1 {
startMonoTime @48 :UInt64;
longitudinalPlanMonoTime @28 :UInt64;
lateralPlanMonoTime @50 :UInt64;
state @31 :OpenpilotState;
enabled @19 :Bool;
active @36 :Bool;
experimentalMode @64 :Bool;
personality @66 :LongitudinalPersonality;
longControlState @30 :Car.CarControl.Actuators.LongControlState;
vPid @2 :Float32;
vTargetLead @3 :Float32;
vCruise @22 :Float32; # actual set speed
vCruiseCluster @63 :Float32; # set speed to display in the UI
upAccelCmd @4 :Float32;
uiAccelCmd @5 :Float32;
ufAccelCmd @33 :Float32;
aTarget @35 :Float32;
curvature @37 :Float32; # path curvature from vehicle model
desiredCurvature @61 :Float32; # lag adjusted curvatures used by lateral controllers
forceDecel @51 :Bool;
# UI alerts
alertText1 @3 :Text;
alertText2 @4 :Text;
alertStatus @5 :AlertStatus;
alertSize @6 :AlertSize;
alertType @7 :Text;
alertSound @8 :Car.CarControl.HUDControl.AudibleAlert;
alertHudVisual @12 :Car.CarControl.HUDControl.VisualAlert;
alertText1 @24 :Text;
alertText2 @25 :Text;
alertStatus @38 :AlertStatus;
alertSize @39 :AlertSize;
alertBlinkingRate @42 :Float32;
alertType @44 :Text;
alertSound @56 :Car.CarControl.HUDControl.AudibleAlert;
engageable @41 :Bool; # can OP be engaged?
# configurable driving settings
experimentalMode @10 :Bool;
personality @11 :LongitudinalPersonality;
cumLagMs @15 :Float32;
lateralControlState :union {
indiState @52 :LateralINDIState;
pidState @53 :LateralPIDState;
angleState @58 :LateralAngleState;
debugState @59 :LateralDebugState;
torqueState @60 :LateralTorqueState;
curvatureStateDEPRECATED @65 :LateralCurvatureState;
lqrStateDEPRECATED @55 :LateralLQRState;
}
enum OpenpilotState @0xdbe58b96d2d1ac61 {
disabled @0;
@ -840,41 +741,17 @@ struct SelfdriveState {
overriding @4; # superset of overriding with steering or accelerator
}
enum AlertStatus @0xa0d0dcd113193c62 {
normal @0;
userPrompt @1;
critical @2;
enum AlertStatus {
normal @0; # low priority alert for user's convenience
userPrompt @1; # mid priority alert that might require user intervention
critical @2; # high priority alert that needs immediate user intervention
}
enum AlertSize @0xe98bb99d6e985f64 {
none @0;
small @1;
mid @2;
full @3;
}
}
struct ControlsState @0x97ff69c53601abf1 {
longitudinalPlanMonoTime @28 :UInt64;
lateralPlanMonoTime @50 :UInt64;
longControlState @30 :Car.CarControl.Actuators.LongControlState;
upAccelCmd @4 :Float32;
uiAccelCmd @5 :Float32;
ufAccelCmd @33 :Float32;
curvature @37 :Float32; # path curvature from vehicle model
desiredCurvature @61 :Float32; # lag adjusted curvatures used by lateral controllers
forceDecel @51 :Bool;
lateralControlState :union {
pidState @53 :LateralPIDState;
angleState @58 :LateralAngleState;
debugState @59 :LateralDebugState;
torqueState @60 :LateralTorqueState;
curvatureStateDEPRECATED @65 :LateralCurvatureState;
lqrStateDEPRECATED @55 :LateralLQRState;
indiStateDEPRECATED @52 :LateralINDIState;
enum AlertSize {
none @0; # don't display the alert
small @1; # small box
mid @2; # mid screen
full @3; # full screen
}
struct LateralINDIState {
@ -918,6 +795,8 @@ struct ControlsState @0x97ff69c53601abf1 {
saturated @7 :Bool;
actualLateralAccel @9 :Float32;
desiredLateralAccel @10 :Float32;
desiredLateralJerk @11 :Float32;
version @12 :Int32;
}
struct LateralLQRState {
@ -989,26 +868,6 @@ struct ControlsState @0x97ff69c53601abf1 {
canMonoTimesDEPRECATED @21 :List(UInt64);
desiredCurvatureRateDEPRECATED @62 :Float32;
canErrorCounterDEPRECATED @57 :UInt32;
vPidDEPRECATED @2 :Float32;
alertBlinkingRateDEPRECATED @42 :Float32;
alertText1DEPRECATED @24 :Text;
alertText2DEPRECATED @25 :Text;
alertStatusDEPRECATED @38 :SelfdriveState.AlertStatus;
alertSizeDEPRECATED @39 :SelfdriveState.AlertSize;
alertTypeDEPRECATED @44 :Text;
alertSound2DEPRECATED @56 :Car.CarControl.HUDControl.AudibleAlert;
engageableDEPRECATED @41 :Bool; # can OP be engaged?
stateDEPRECATED @31 :SelfdriveState.OpenpilotState;
enabledDEPRECATED @19 :Bool;
activeDEPRECATED @36 :Bool;
experimentalModeDEPRECATED @64 :Bool;
personalityDEPRECATED @66 :LongitudinalPersonality;
vCruiseDEPRECATED @22 :Float32; # actual set speed
vCruiseClusterDEPRECATED @63 :Float32; # set speed to display in the UI
startMonoTimeDEPRECATED @48 :UInt64;
cumLagMsDEPRECATED @15 :Float32;
aTargetDEPRECATED @35 :Float32;
vTargetLeadDEPRECATED @3 :Float32;
}
struct DrivingModelData {
@ -1061,6 +920,7 @@ struct ModelDataV2 {
frameDropPerc @2 :Float32;
timestampEof @3 :UInt64;
modelExecutionTime @15 :Float32;
gpuExecutionTime @17 :Float32;
rawPredictions @16 :Data;
# predicted future position, orientation, etc..
@ -1085,15 +945,14 @@ struct ModelDataV2 {
confidence @23: ConfidenceClass;
# Model perceived motion
temporalPoseDEPRECATED @21 :Pose;
temporalPose @21 :Pose;
navEnabled @22 :Bool;
locationMonoTime @24 :UInt64;
# e2e lateral planner
action @26: Action;
gpuExecutionTimeDEPRECATED @17 :Float32;
navEnabledDEPRECATED @22 :Bool;
locationMonoTimeDEPRECATED @24 :UInt64;
lateralPlannerSolutionDEPRECATED @25: LateralPlannerSolution;
action @26: Action;
struct LeadDataV2 {
prob @0 :Float32; # probability that car is your lead at time t
@ -1228,14 +1087,6 @@ struct AndroidLogEntry {
message @6 :Text;
}
struct DriverAssistance {
# Lane Departure Warnings
leftLaneDeparture @0 :Bool;
rightLaneDeparture @1 :Bool;
# FCW, AEB, etc. will go here
}
struct LongitudinalPlan @0xe00b5b3eba12876c {
modelMonoTime @9 :UInt64;
hasLead @7 :Bool;
@ -1287,7 +1138,7 @@ struct LongitudinalPlan @0xe00b5b3eba12876c {
radarValidDEPRECATED @28 :Bool;
radarCanErrorDEPRECATED @30 :Bool;
commIssueDEPRECATED @31 :Bool;
eventsDEPRECATED @13 :List(Car.OnroadEventDEPRECATED);
eventsDEPRECATED @13 :List(Car.CarEvent);
gpsTrajectoryDEPRECATED @12 :GpsTrajectory;
gpsPlannerActiveDEPRECATED @19 :Bool;
personalityDEPRECATED @36 :LongitudinalPersonality;
@ -1421,7 +1272,7 @@ struct LivePose {
posenetOK @5 :Bool = false;
sensorsOK @6 :Bool = false;
debugFilterState @7 :FilterState;
filterState @7 :FilterState;
struct XYZMeasurement {
x @0 :Float32;
@ -1437,14 +1288,6 @@ struct LivePose {
value @0 : List(Float64);
std @1 : List(Float64);
valid @2 : Bool;
observations @3 :List(Observation);
struct Observation {
kind @0 :Int32;
value @1 :List(Float32);
error @2 :List(Float32);
}
}
}
@ -1593,10 +1436,6 @@ struct UbloxGnss {
svId @0 :UInt8;
gnssId @1 :UInt8;
flagsBitfield @2 :UInt32;
cno @3 :UInt8;
elevationDeg @4 :Int8;
azimuthDeg @5 :Int16;
pseudorangeResidual @6 :Float32;
}
}
@ -2146,10 +1985,12 @@ struct Joystick {
struct DriverStateV2 {
frameId @0 :UInt32;
modelExecutionTime @1 :Float32;
gpuExecutionTime @8 :Float32;
dspExecutionTime @2 :Float32;
rawPredictions @3 :Data;
poorVisionProb @4 :Float32;
wheelOnRightProb @5 :Float32;
leftDriverData @6 :DriverData;
rightDriverData @7 :DriverData;
@ -2164,13 +2005,10 @@ struct DriverStateV2 {
leftBlinkProb @7 :Float32;
rightBlinkProb @8 :Float32;
sunglassesProb @9 :Float32;
occludedProb @10 :Float32;
readyProb @11 :List(Float32);
notReadyProb @12 :List(Float32);
occludedProbDEPRECATED @10 :Float32;
readyProbDEPRECATED @11 :List(Float32);
}
dspExecutionTimeDEPRECATED @2 :Float32;
poorVisionProbDEPRECATED @4 :Float32;
}
struct DriverStateDEPRECATED @0xb83c6cc593ed0a00 {
@ -2206,7 +2044,7 @@ struct DriverStateDEPRECATED @0xb83c6cc593ed0a00 {
}
struct DriverMonitoringState @0xb83cda094a1da284 {
events @18 :List(OnroadEvent);
events @0 :List(Car.CarEvent);
faceDetected @1 :Bool;
isDistracted @2 :Bool;
distractedType @17 :UInt32;
@ -2225,7 +2063,6 @@ struct DriverMonitoringState @0xb83cda094a1da284 {
isPreviewDEPRECATED @15 :Bool;
rhdCheckedDEPRECATED @5 :Bool;
eventsDEPRECATED @0 :List(Car.OnroadEventDEPRECATED);
}
struct Boot {
@ -2253,20 +2090,9 @@ struct LiveParametersData {
stiffnessFactorStd @12 :Float32;
steerRatioStd @13 :Float32;
roll @14 :Float32;
debugFilterState @16 :FilterState;
angleOffsetValid @17 :Bool = true;
angleOffsetAverageValid @18 :Bool = true;
steerRatioValid @19 :Bool = true;
stiffnessFactorValid @20 :Bool = true;
filterState @15 :LiveLocationKalman.Measurement;
yawRateDEPRECATED @7 :Float32;
filterStateDEPRECATED @15 :LiveLocationKalman.Measurement;
struct FilterState {
value @0 : List(Float64);
std @1 : List(Float64);
}
}
struct LiveTorqueParametersData {
@ -2283,7 +2109,6 @@ struct LiveTorqueParametersData {
points @10 :List(List(Float32));
version @11 :Int32;
useParams @12 :Bool;
calPerc @13 :Int8;
}
struct LiveDelayData {
@ -2294,7 +2119,6 @@ struct LiveDelayData {
lateralDelayEstimate @3 :Float32;
lateralDelayEstimateStd @5 :Float32;
points @4 :List(Float32);
calPerc @6 :Int8;
enum Status {
unestimated @0;
@ -2464,40 +2288,16 @@ struct EncodeData {
height @5 :UInt32;
}
struct DebugAlert {
alertText1 @0 :Text;
alertText2 @1 :Text;
struct UserFlag {
}
struct UserBookmark @0xfe346a9de48d9b50 {
}
struct SoundPressure @0xdc24138990726023 {
struct Microphone {
soundPressure @0 :Float32;
# uncalibrated, A-weighted
soundPressureWeighted @3 :Float32;
soundPressureWeightedDb @1 :Float32;
filteredSoundPressureWeightedDbDEPRECATED @2 :Float32;
}
struct AudioData {
data @0 :Data;
sampleRate @1 :UInt32;
}
struct AudioFeedback {
audio @0 :AudioData;
blockNum @1 :UInt16;
}
struct Touch {
sec @0 :Int64;
usec @1 :Int64;
type @2 :UInt8;
code @3 :Int32;
value @4 :Int32;
filteredSoundPressureWeightedDb @2 :Float32;
}
struct Event {
@ -2516,7 +2316,6 @@ struct Event {
gpsNMEA @3 :GPSNMEAData;
can @5 :List(CanData);
controlsState @7 :ControlsState;
selfdriveState @130 :SelfdriveState;
gyroscope @99 :SensorEventData;
gyroscope2 @100 :SensorEventData;
accelerometer @98 :SensorEventData;
@ -2528,14 +2327,14 @@ struct Event {
pandaStates @81 :List(PandaState);
peripheralState @80 :PeripheralState;
radarState @13 :RadarState;
liveTracks @131 :Car.RadarData;
liveTracks @16 :List(LiveTracks);
sendcan @17 :List(CanData);
liveCalibration @19 :LiveCalibrationData;
carState @22 :Car.CarState;
carControl @23 :Car.CarControl;
carOutput @127 :Car.CarOutput;
longitudinalPlan @24 :LongitudinalPlan;
driverAssistance @132 :DriverAssistance;
uiPlan @106 :UiPlan;
ubloxGnss @34 :UbloxGnss;
ubloxRaw @39 :Data;
qcomGnss @31 :QcomGnss;
@ -2544,12 +2343,13 @@ struct Event {
gnssMeasurements @91 :GnssMeasurements;
liveParameters @61 :LiveParametersData;
liveTorqueParameters @94 :LiveTorqueParametersData;
liveDelay @146 : LiveDelayData;
liveDelay @130 : LiveDelayData;
cameraOdometry @63 :CameraOdometry;
thumbnail @66: Thumbnail;
onroadEvents @134: List(OnroadEvent);
onroadEvents @68: List(Car.CarEvent);
carParams @69: Car.CarParams;
driverMonitoringState @71: DriverMonitoringState;
liveLocationKalman @72 :LiveLocationKalman;
livePose @129 :LivePose;
modelV2 @75 :ModelDataV2;
drivingModelData @128 :DrivingModelData;
@ -2569,8 +2369,7 @@ struct Event {
livestreamDriverEncodeIdx @119 :EncodeIndex;
# microphone data
soundPressure @103 :SoundPressure;
rawAudioData @147 :AudioData;
microphone @103 :Microphone;
# systems stuff
androidLog @20 :AndroidLogEntry;
@ -2582,9 +2381,6 @@ struct Event {
logMessage @18 :Text;
errorLogMessage @85 :Text;
# touch frame
touch @135 :List(Touch);
# navigation
navInstruction @82 :NavInstruction;
navRoute @83 :NavRoute;
@ -2592,62 +2388,41 @@ struct Event {
mapRenderState @105: MapRenderState;
# UI services
userFlag @93 :UserFlag;
uiDebug @102 :UIDebug;
# driving feedback
userBookmark @93 :UserBookmark;
bookmarkButton @148 :UserBookmark;
audioFeedback @149 :AudioFeedback;
# *********** debug ***********
testJoystick @52 :Joystick;
roadEncodeData @86 :EncodeData;
driverEncodeData @87 :EncodeData;
wideRoadEncodeData @88 :EncodeData;
qRoadEncodeData @89 :EncodeData;
alertDebug @133 :DebugAlert;
livestreamRoadEncodeData @120 :EncodeData;
livestreamWideRoadEncodeData @121 :EncodeData;
livestreamDriverEncodeData @122 :EncodeData;
# *********** Custom: reserved for forks ***********
# DO change the name of the field
# DON'T change anything after the "@"
customReservedRawData0 @124 :Data;
customReservedRawData1 @125 :Data;
customReservedRawData2 @126 :Data;
# DO change the name of the field and struct
# DON'T change the ID (e.g. @107)
# DON'T change which struct it points to
dpControlsState @107 :Custom.DpControlsState;
modelExt @108 :Custom.ModelExt;
customReserved2 @109 :Custom.CustomReserved2;
customReserved3 @110 :Custom.CustomReserved3;
customReserved4 @111 :Custom.CustomReserved4;
customReserved5 @112 :Custom.CustomReserved5;
customReserved6 @113 :Custom.CustomReserved6;
customReserved7 @114 :Custom.CustomReserved7;
customReserved8 @115 :Custom.CustomReserved8;
customReserved9 @116 :Custom.CustomReserved9;
customReserved10 @136 :Custom.CustomReserved10;
customReserved11 @137 :Custom.CustomReserved11;
customReserved12 @138 :Custom.CustomReserved12;
customReserved13 @139 :Custom.CustomReserved13;
customReserved14 @140 :Custom.CustomReserved14;
customReserved15 @141 :Custom.CustomReserved15;
customReserved16 @142 :Custom.CustomReserved16;
customReserved17 @143 :Custom.CustomReserved17;
customReserved18 @144 :Custom.CustomReserved18;
customReserved19 @145 :Custom.CustomReserved19;
# *********** Custom: reserved for forks ***********
frogpilotCarControl @107 :Custom.FrogPilotCarControl;
frogpilotCarParams @108 :Custom.FrogPilotCarParams;
frogpilotCarState @109 :Custom.FrogPilotCarState;
frogpilotControlsState @110 :Custom.FrogPilotControlsState;
frogpilotDeviceState @111 :Custom.FrogPilotDeviceState;
frogpilotModelV2 @112 :Custom.FrogPilotModelDataV2;
frogpilotNavigation @113 :Custom.FrogPilotNavigation;
frogpilotOnroadEvents @114: List(Custom.FrogPilotCarEvent);
frogpilotPlan @115 :Custom.FrogPilotPlan;
frogpilotRadarState @116 :Custom.FrogPilotRadarState;
# *********** legacy + deprecated ***********
model @9 :Legacy.ModelData; # TODO: rename modelV2 and mark this as deprecated
liveMpcDEPRECATED @36 :LiveMpcData;
liveLongitudinalMpcDEPRECATED @37 :LiveLongitudinalMpcData;
liveLocationKalmanLegacyDEPRECATED @51 :Legacy.LiveLocationData;
liveLocationKalmanDEPRECATED @51 :Legacy.LiveLocationData;
orbslamCorrectionDEPRECATED @45 :Legacy.OrbslamCorrection;
liveUIDEPRECATED @14 :Legacy.LiveUI;
sensorEventDEPRECATED @4 :SensorEventData;
@ -2682,10 +2457,6 @@ struct Event {
driverStateDEPRECATED @59 :DriverStateDEPRECATED;
sensorEventsDEPRECATED @11 :List(SensorEventData);
lateralPlanDEPRECATED @64 :LateralPlan;
navModelDEPRECATED @104 :NavModelData;
uiPlanDEPRECATED @106 :UiPlan;
liveLocationKalmanDEPRECATED @72 :LiveLocationKalman;
liveTracksDEPRECATED @16 :List(LiveTracksDEPRECATED);
onroadEventsDEPRECATED @68: List(Car.OnroadEventDEPRECATED);
navModel @104 :NavModelData;
}
}

49
cereal/maptile.capnp Normal file
View File

@ -0,0 +1,49 @@
using Cxx = import "./include/c++.capnp";
$Cxx.namespace("cereal");
@0xa086df597ef5d7a0;
# Geometry
struct Point {
x @0: Float64;
y @1: Float64;
z @2: Float64;
}
struct PolyLine {
points @0: List(Point);
}
# Map features
struct Lane {
id @0 :Text;
leftBoundary @1 :LaneBoundary;
rightBoundary @2 :LaneBoundary;
leftAdjacentId @3 :Text;
rightAdjacentId @4 :Text;
inboundIds @5 :List(Text);
outboundIds @6 :List(Text);
struct LaneBoundary {
polyLine @0 :PolyLine;
startHeading @1 :Float32; # WRT north
}
}
# Map tiles
struct TileSummary {
version @0 :Text;
updatedAt @1 :UInt64; # Millis since epoch
level @2 :UInt8;
x @3 :UInt16;
y @4 :UInt16;
}
struct MapTile {
summary @0 :TileSummary;
lanes @1 :List(Lane);
}

10
cereal/messaging/.gitignore vendored Normal file
View File

@ -0,0 +1,10 @@
demo
bridge
test_runner
*.o
*.os
*.d
*.a
*.so
messaging_pyx.cpp
build/

View File

@ -2,28 +2,23 @@
from msgq.ipc_pyx import Context, Poller, SubSocket, PubSocket, SocketEventHandle, toggle_fake_events, \
set_fake_prefix, get_fake_prefix, delete_fake_prefix, wait_for_one_event
from msgq.ipc_pyx import MultiplePublishersError, IpcError
from msgq import fake_event_handle, pub_sock, sub_sock, drain_sock_raw
import msgq
from msgq import fake_event_handle, pub_sock, sub_sock, drain_sock_raw, context
import os
import capnp
import time
from typing import Optional, List, Union, Dict
from typing import Optional, List, Union, Dict, Deque
from collections import deque
from cereal import log
from cereal.services import SERVICE_LIST
from openpilot.common.util import MovingAverage
NO_TRAVERSAL_LIMIT = 2**64-1
def reset_context():
msgq.context = Context()
def log_from_bytes(dat: bytes, struct: capnp.lib.capnp._StructModule = log.Event) -> capnp.lib.capnp._DynamicStructReader:
with struct.from_bytes(dat, traversal_limit_in_words=NO_TRAVERSAL_LIMIT) as msg:
def log_from_bytes(dat: bytes) -> capnp.lib.capnp._DynamicStructReader:
with log.Event.from_bytes(dat, traversal_limit_in_words=NO_TRAVERSAL_LIMIT) as msg:
return msg
@ -92,71 +87,26 @@ def recv_one_retry(sock: SubSocket) -> capnp.lib.capnp._DynamicStructReader:
return log_from_bytes(dat)
class FrequencyTracker:
def __init__(self, service_freq: float, update_freq: float, is_poll: bool):
freq = max(min(service_freq, update_freq), 1.)
if is_poll:
min_freq = max_freq = freq
else:
max_freq = min(freq, update_freq)
if service_freq >= 2 * update_freq:
min_freq = update_freq
elif update_freq >= 2* service_freq:
min_freq = freq
else:
min_freq = min(freq, freq / 2.)
self.min_freq = min_freq * 0.8
self.max_freq = max_freq * 1.2
self.avg_dt = MovingAverage(int(10 * freq))
self.recent_avg_dt = MovingAverage(int(freq))
self.prev_time = 0.0
def record_recv_time(self, cur_time: float) -> None:
# TODO: Handle case where cur_time is less than prev_time
if self.prev_time > 1e-5:
dt = cur_time - self.prev_time
self.avg_dt.add_value(dt)
self.recent_avg_dt.add_value(dt)
self.prev_time = cur_time
@property
def valid(self) -> bool:
if self.avg_dt.count == 0:
return False
avg_freq = 1.0 / self.avg_dt.get_average()
if self.min_freq <= avg_freq <= self.max_freq:
return True
avg_freq_recent = 1.0 / self.recent_avg_dt.get_average()
return self.min_freq <= avg_freq_recent <= self.max_freq
class SubMaster:
def __init__(self, services: List[str], poll: Optional[str] = None,
ignore_alive: Optional[List[str]] = None, ignore_avg_freq: Optional[List[str]] = None,
ignore_valid: Optional[List[str]] = None, addr: str = "127.0.0.1", frequency: Optional[float] = None):
self.frame = -1
self.services = services
self.seen = {s: False for s in services}
self.updated = {s: False for s in services}
self.recv_time = {s: 0. for s in services}
self.recv_frame = {s: 0 for s in services}
self.alive = {s: False for s in services}
self.freq_ok = {s: False for s in services}
self.recv_dts: Dict[str, Deque[float]] = {}
self.sock = {}
self.data = {}
self.logMonoTime = {s: 0 for s in services}
self.valid = {}
self.logMonoTime = {}
# zero-frequency / on-demand services are always alive and presumed valid; all others must pass checks
on_demand = {s: SERVICE_LIST[s].frequency <= 1e-5 for s in services}
self.static_freq_services = set(s for s in services if not on_demand[s])
self.alive = {s: on_demand[s] for s in services}
self.freq_ok = {s: on_demand[s] for s in services}
self.valid = {s: on_demand[s] for s in services}
self.max_freq = {}
self.min_freq = {}
self.freq_tracker: Dict[str, FrequencyTracker] = {}
self.poller = Poller()
polled_services = set([poll, ] if poll is not None else services)
self.non_polled_services = set(services) - polled_services
@ -181,7 +131,24 @@ class SubMaster:
data = new_message(s, 0) # lists
self.data[s] = getattr(data.as_reader(), s)
self.freq_tracker[s] = FrequencyTracker(SERVICE_LIST[s].frequency, self.update_freq, s == poll)
self.logMonoTime[s] = 0
self.valid[s] = True # FIXME: this should default to False
freq = max(min([SERVICE_LIST[s].frequency, self.update_freq]), 1.)
if s == poll:
max_freq = freq
min_freq = freq
else:
max_freq = min(freq, self.update_freq)
if SERVICE_LIST[s].frequency >= 2*self.update_freq:
min_freq = self.update_freq
elif self.update_freq >= 2*SERVICE_LIST[s].frequency:
min_freq = freq
else:
min_freq = min(freq, freq / 2.)
self.max_freq[s] = max_freq*1.2
self.min_freq[s] = min_freq*0.8
self.recv_dts[s] = deque(maxlen=int(10*freq))
def __getitem__(self, s: str) -> capnp.lib.capnp._DynamicStructReader:
return self.data[s]
@ -201,7 +168,7 @@ class SubMaster:
def update_msgs(self, cur_time: float, msgs: List[capnp.lib.capnp._DynamicStructReader]) -> None:
self.frame += 1
self.updated = dict.fromkeys(self.services, False)
self.updated = dict.fromkeys(self.updated, False)
for msg in msgs:
if msg is None:
continue
@ -210,26 +177,54 @@ class SubMaster:
self.seen[s] = True
self.updated[s] = True
self.freq_tracker[s].record_recv_time(cur_time)
if self.recv_time[s] > 1e-5:
self.recv_dts[s].append(cur_time - self.recv_time[s])
self.recv_time[s] = cur_time
self.recv_frame[s] = self.frame
self.data[s] = getattr(msg, s)
self.logMonoTime[s] = msg.logMonoTime
self.valid[s] = msg.valid
for s in self.static_freq_services:
# alive if delay is within 10x the expected frequency; checks relaxed in simulator
self.alive[s] = (cur_time - self.recv_time[s]) < (10. / SERVICE_LIST[s].frequency) or (self.seen[s] and self.simulation)
self.freq_ok[s] = self.freq_tracker[s].valid or self.simulation
for s in self.data:
if SERVICE_LIST[s].frequency > 1e-5 and not self.simulation:
# alive if delay is within 10x the expected frequency
self.alive[s] = (cur_time - self.recv_time[s]) < (10. / SERVICE_LIST[s].frequency)
# check average frequency; slow to fall, quick to recover
dts = self.recv_dts[s]
assert dts.maxlen is not None
recent_dts = list(dts)[-int(dts.maxlen / 10):]
try:
avg_freq = 1 / (sum(dts) / len(dts))
avg_freq_recent = 1 / (sum(recent_dts) / len(recent_dts))
except ZeroDivisionError:
avg_freq = 0
avg_freq_recent = 0
avg_freq_ok = self.min_freq[s] <= avg_freq <= self.max_freq[s]
recent_freq_ok = self.min_freq[s] <= avg_freq_recent <= self.max_freq[s]
self.freq_ok[s] = avg_freq_ok or recent_freq_ok
else:
self.freq_ok[s] = True
if self.simulation:
self.alive[s] = self.seen[s] # alive is defined as seen when simulation flag set
else:
self.alive[s] = True
def all_alive(self, service_list: Optional[List[str]] = None) -> bool:
return all(self.alive[s] for s in (service_list or self.services) if s not in self.ignore_alive)
if service_list is None:
service_list = list(self.sock.keys())
return all(self.alive[s] for s in service_list if s not in self.ignore_alive)
def all_freq_ok(self, service_list: Optional[List[str]] = None) -> bool:
return all(self.freq_ok[s] for s in (service_list or self.services) if self._check_avg_freq(s))
if service_list is None:
service_list = list(self.sock.keys())
return all(self.freq_ok[s] for s in service_list if self._check_avg_freq(s))
def all_valid(self, service_list: Optional[List[str]] = None) -> bool:
return all(self.valid[s] for s in (service_list or self.services) if s not in self.ignore_valid)
if service_list is None:
service_list = list(self.sock.keys())
return all(self.valid[s] for s in service_list if s not in self.ignore_valid)
def all_checks(self, service_list: Optional[List[str]] = None) -> bool:
return self.all_alive(service_list) and self.all_freq_ok(service_list) and self.all_valid(service_list)

BIN
cereal/messaging/bridge Executable file

Binary file not shown.

View File

@ -1,17 +1,32 @@
#include <algorithm>
#include <cassert>
#include <csignal>
#include <iostream>
#include <map>
#include <string>
typedef void (*sighandler_t)(int sig);
#include "cereal/messaging/msgq_to_zmq.h"
#include "cereal/services.h"
#include "common/util.h"
#include "msgq/impl_msgq.h"
#include "msgq/impl_zmq.h"
ExitHandler do_exit;
std::atomic<bool> do_exit = false;
static void set_do_exit(int sig) {
do_exit = true;
}
static std::vector<std::string> get_services(const std::string &whitelist_str, bool zmq_to_msgq) {
void sigpipe_handler(int sig) {
assert(sig == SIGPIPE);
std::cout << "SIGPIPE received" << std::endl;
}
static std::vector<std::string> get_services(std::string whitelist_str, bool zmq_to_msgq) {
std::vector<std::string> service_list;
for (const auto& it : services) {
std::string name = it.second.name;
bool in_whitelist = whitelist_str.find(name) != std::string::npos;
if (zmq_to_msgq && !in_whitelist) {
if (name == "plusFrame" || name == "uiLayoutState" || (zmq_to_msgq && !in_whitelist)) {
continue;
}
service_list.push_back(name);
@ -19,22 +34,41 @@ static std::vector<std::string> get_services(const std::string &whitelist_str, b
return service_list;
}
void msgq_to_zmq(const std::vector<std::string> &endpoints, const std::string &ip) {
MsgqToZmq bridge;
bridge.run(endpoints, ip);
int main(int argc, char** argv) {
signal(SIGPIPE, (sighandler_t)sigpipe_handler);
signal(SIGINT, (sighandler_t)set_do_exit);
signal(SIGTERM, (sighandler_t)set_do_exit);
bool zmq_to_msgq = argc > 2;
std::string ip = zmq_to_msgq ? argv[1] : "127.0.0.1";
std::string whitelist_str = zmq_to_msgq ? std::string(argv[2]) : "";
Poller *poller;
Context *pub_context;
Context *sub_context;
if (zmq_to_msgq) { // republishes zmq debugging messages as msgq
poller = new ZMQPoller();
pub_context = new MSGQContext();
sub_context = new ZMQContext();
} else {
poller = new MSGQPoller();
pub_context = new ZMQContext();
sub_context = new MSGQContext();
}
void zmq_to_msgq(const std::vector<std::string> &endpoints, const std::string &ip) {
auto poller = std::make_unique<ZMQPoller>();
auto pub_context = std::make_unique<MSGQContext>();
auto sub_context = std::make_unique<ZMQContext>();
std::map<SubSocket*, PubSocket*> sub2pub;
for (auto endpoint : endpoints) {
auto pub_sock = new MSGQPubSocket();
auto sub_sock = new ZMQSubSocket();
pub_sock->connect(pub_context.get(), endpoint);
sub_sock->connect(sub_context.get(), endpoint, ip, false);
for (auto endpoint : get_services(whitelist_str, zmq_to_msgq)) {
PubSocket * pub_sock;
SubSocket * sub_sock;
if (zmq_to_msgq) {
pub_sock = new MSGQPubSocket();
sub_sock = new ZMQSubSocket();
} else {
pub_sock = new ZMQPubSocket();
sub_sock = new MSGQSubSocket();
}
pub_sock->connect(pub_context, endpoint);
sub_sock->connect(sub_context, endpoint, ip, false);
poller->registerSocket(sub_sock);
sub2pub[sub_sock] = pub_sock;
@ -42,30 +76,17 @@ void zmq_to_msgq(const std::vector<std::string> &endpoints, const std::string &i
while (!do_exit) {
for (auto sub_sock : poller->poll(100)) {
std::unique_ptr<Message> msg(sub_sock->receive(true));
if (msg) {
sub2pub[sub_sock]->sendMessage(msg.get());
}
}
}
Message * msg = sub_sock->receive();
if (msg == NULL) continue;
int ret;
do {
ret = sub2pub[sub_sock]->sendMessage(msg);
} while (ret == -1 && errno == EINTR && !do_exit);
assert(ret >= 0 || do_exit);
delete msg;
// Clean up allocated sockets
for (auto &[sub_sock, pub_sock] : sub2pub) {
delete sub_sock;
delete pub_sock;
if (do_exit) break;
}
}
int main(int argc, char **argv) {
bool is_zmq_to_msgq = argc > 2;
std::string ip = is_zmq_to_msgq ? argv[1] : "127.0.0.1";
std::string whitelist_str = is_zmq_to_msgq ? std::string(argv[2]) : "";
std::vector<std::string> endpoints = get_services(whitelist_str, is_zmq_to_msgq);
if (is_zmq_to_msgq) {
zmq_to_msgq(endpoints, ip);
} else {
msgq_to_zmq(endpoints, ip);
}
return 0;
}

View File

@ -5,13 +5,20 @@
#include <string>
#include <vector>
#include <utility>
#include <time.h>
#include <capnp/serialize.h>
#include "cereal/gen/cpp/log.capnp.h"
#include "common/timing.h"
#include "msgq/ipc.h"
#ifdef __APPLE__
#define CLOCK_BOOTTIME CLOCK_MONOTONIC
#endif
#define MSG_MULTIPLE_PUBLISHERS 100
class SubMaster {
public:
SubMaster(const std::vector<const char *> &service_list, const std::vector<const char *> &poll = {},
@ -46,7 +53,10 @@ public:
cereal::Event::Builder initEvent(bool valid = true) {
cereal::Event::Builder event = initRoot<cereal::Event>();
event.setLogMonoTime(nanos_since_boot());
struct timespec t;
clock_gettime(CLOCK_BOOTTIME, &t);
uint64_t current_time = t.tv_sec * 1000000000ULL + t.tv_nsec;
event.setLogMonoTime(current_time);
event.setValid(valid);
return event;
}

View File

@ -1,144 +0,0 @@
#include "cereal/messaging/msgq_to_zmq.h"
#include <cassert>
#include "common/util.h"
extern ExitHandler do_exit;
// Max messages to process per socket per poll
constexpr int MAX_MESSAGES_PER_SOCKET = 50;
static std::string recv_zmq_msg(void *sock) {
zmq_msg_t msg;
zmq_msg_init(&msg);
std::string ret;
if (zmq_msg_recv(&msg, sock, 0) > 0) {
ret.assign((char *)zmq_msg_data(&msg), zmq_msg_size(&msg));
}
zmq_msg_close(&msg);
return ret;
}
void MsgqToZmq::run(const std::vector<std::string> &endpoints, const std::string &ip) {
zmq_context = std::make_unique<ZMQContext>();
msgq_context = std::make_unique<MSGQContext>();
// Create ZMQPubSockets for each endpoint
for (const auto &endpoint : endpoints) {
auto &socket_pair = socket_pairs.emplace_back();
socket_pair.endpoint = endpoint;
socket_pair.pub_sock = std::make_unique<ZMQPubSocket>();
int ret = socket_pair.pub_sock->connect(zmq_context.get(), endpoint);
if (ret != 0) {
printf("Failed to create ZMQ publisher for [%s]: %s\n", endpoint.c_str(), zmq_strerror(zmq_errno()));
return;
}
}
// Start ZMQ monitoring thread to monitor socket events
std::thread thread(&MsgqToZmq::zmqMonitorThread, this);
// Main loop for processing messages
while (!do_exit) {
{
std::unique_lock lk(mutex);
cv.wait(lk, [this]() { return do_exit || !sub2pub.empty(); });
if (do_exit) break;
for (auto sub_sock : msgq_poller->poll(100)) {
// Process messages for each socket
ZMQPubSocket *pub_sock = sub2pub.at(sub_sock);
for (int i = 0; i < MAX_MESSAGES_PER_SOCKET; ++i) {
auto msg = std::unique_ptr<Message>(sub_sock->receive(true));
if (!msg) break;
while (pub_sock->sendMessage(msg.get()) == -1) {
if (errno != EINTR) break;
}
}
}
}
util::sleep_for(1); // Give zmqMonitorThread a chance to acquire the mutex
}
thread.join();
}
void MsgqToZmq::zmqMonitorThread() {
std::vector<zmq_pollitem_t> pollitems;
// Set up ZMQ monitor for each pub socket
for (int i = 0; i < socket_pairs.size(); ++i) {
std::string addr = "inproc://op-bridge-monitor-" + std::to_string(i);
zmq_socket_monitor(socket_pairs[i].pub_sock->sock, addr.c_str(), ZMQ_EVENT_ACCEPTED | ZMQ_EVENT_DISCONNECTED);
void *monitor_socket = zmq_socket(zmq_context->getRawContext(), ZMQ_PAIR);
zmq_connect(monitor_socket, addr.c_str());
pollitems.emplace_back(zmq_pollitem_t{.socket = monitor_socket, .events = ZMQ_POLLIN});
}
while (!do_exit) {
int ret = zmq_poll(pollitems.data(), pollitems.size(), 1000);
if (ret < 0) {
if (errno == EINTR) {
// Due to frequent EINTR signals from msgq, introduce a brief delay (200 ms)
// to reduce CPU usage during retry attempts.
util::sleep_for(200);
}
continue;
}
for (int i = 0; i < pollitems.size(); ++i) {
if (pollitems[i].revents & ZMQ_POLLIN) {
// First frame in message contains event number and value
std::string frame = recv_zmq_msg(pollitems[i].socket);
if (frame.empty()) continue;
uint16_t event_type = *(uint16_t *)(frame.data());
// Second frame in message contains event address
frame = recv_zmq_msg(pollitems[i].socket);
if (frame.empty()) continue;
std::unique_lock lk(mutex);
auto &pair = socket_pairs[i];
if (event_type & ZMQ_EVENT_ACCEPTED) {
printf("socket [%s] connected\n", pair.endpoint.c_str());
if (++pair.connected_clients == 1) {
// Create new MSGQ subscriber socket and map to ZMQ publisher
pair.sub_sock = std::make_unique<MSGQSubSocket>();
pair.sub_sock->connect(msgq_context.get(), pair.endpoint, "127.0.0.1");
sub2pub[pair.sub_sock.get()] = pair.pub_sock.get();
registerSockets();
}
} else if (event_type & ZMQ_EVENT_DISCONNECTED) {
printf("socket [%s] disconnected\n", pair.endpoint.c_str());
if (pair.connected_clients == 0 || --pair.connected_clients == 0) {
// Remove MSGQ subscriber socket from mapping and reset it
sub2pub.erase(pair.sub_sock.get());
pair.sub_sock.reset(nullptr);
registerSockets();
}
}
cv.notify_one();
}
}
}
// Clean up monitor sockets
for (int i = 0; i < pollitems.size(); ++i) {
zmq_socket_monitor(socket_pairs[i].pub_sock->sock, nullptr, 0);
zmq_close(pollitems[i].socket);
}
cv.notify_one();
}
void MsgqToZmq::registerSockets() {
msgq_poller = std::make_unique<MSGQPoller>();
for (const auto &socket_pair : socket_pairs) {
if (socket_pair.sub_sock) {
msgq_poller->registerSocket(socket_pair.sub_sock.get());
}
}
}

View File

@ -1,37 +0,0 @@
#pragma once
#include <condition_variable>
#include <map>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
#define private public
#include "msgq/impl_msgq.h"
#include "msgq/impl_zmq.h"
class MsgqToZmq {
public:
MsgqToZmq() {}
void run(const std::vector<std::string> &endpoints, const std::string &ip);
protected:
void registerSockets();
void zmqMonitorThread();
struct SocketPair {
std::string endpoint;
std::unique_ptr<ZMQPubSocket> pub_sock;
std::unique_ptr<MSGQSubSocket> sub_sock;
int connected_clients = 0;
};
std::unique_ptr<MSGQContext> msgq_context;
std::unique_ptr<ZMQContext> zmq_context;
std::mutex mutex;
std::condition_variable cv;
std::unique_ptr<MSGQPoller> msgq_poller;
std::map<SubSocket *, ZMQPubSocket *> sub2pub;
std::vector<SocketPair> socket_pairs;
};

View File

@ -1,3 +1,4 @@
#include <time.h>
#include <assert.h>
#include <stdlib.h>
#include <string>
@ -6,8 +7,15 @@
#include "cereal/services.h"
#include "cereal/messaging/messaging.h"
const bool SIMULATION = (getenv("SIMULATION") != nullptr) && (std::string(getenv("SIMULATION")) == "1");
static inline uint64_t nanos_since_boot() {
struct timespec t;
clock_gettime(CLOCK_BOOTTIME, &t);
return t.tv_sec * 1000000000ULL + t.tv_nsec;
}
static inline bool inList(const std::vector<const char *> &list, const char *value) {
for (auto &v : list) {
if (strcmp(value, v) == 0) return true;
@ -33,8 +41,8 @@ MessageContext message_context;
struct SubMaster::SubMessage {
std::string name;
SubSocket *socket = nullptr;
float freq = 0.0f;
bool updated = false, alive = false, valid = false, ignore_alive;
int freq = 0;
bool updated = false, alive = false, valid = true, ignore_alive;
uint64_t rcv_time = 0, rcv_frame = 0;
void *allocated_msg_reader = nullptr;
bool is_polled = false;

View File

@ -1,185 +0,0 @@
import os
import capnp
import multiprocessing
import numbers
import random
import threading
import time
from parameterized import parameterized
import pytest
from cereal import log, car
import cereal.messaging as messaging
from cereal.services import SERVICE_LIST
events = [evt for evt in log.Event.schema.union_fields if evt in SERVICE_LIST.keys()]
def random_sock():
return random.choice(events)
def random_socks(num_socks=10):
return list({random_sock() for _ in range(num_socks)})
def random_bytes(length=1000):
return bytes([random.randrange(0xFF) for _ in range(length)])
def zmq_sleep(t=1):
if "ZMQ" in os.environ:
time.sleep(t)
# TODO: this should take any capnp struct and returrn a msg with random populated data
def random_carstate():
fields = ["vEgo", "aEgo", "brake", "steeringAngleDeg"]
msg = messaging.new_message("carState")
cs = msg.carState
for f in fields:
setattr(cs, f, random.random() * 10)
return msg
# TODO: this should compare any capnp structs
def assert_carstate(cs1, cs2):
for f in car.CarState.schema.non_union_fields:
# TODO: check all types
val1, val2 = getattr(cs1, f), getattr(cs2, f)
if isinstance(val1, numbers.Number):
assert val1 == val2, f"{f}: sent '{val1}' vs recvd '{val2}'"
def delayed_send(delay, sock, dat):
def send_func():
sock.send(dat)
threading.Timer(delay, send_func).start()
class TestMessaging:
def setUp(self):
# TODO: ZMQ tests are too slow; all sleeps will need to be
# replaced with logic to block on the necessary condition
if "ZMQ" in os.environ:
pytest.skip()
# ZMQ pub socket takes too long to die
# sleep to prevent multiple publishers error between tests
zmq_sleep()
@parameterized.expand(events)
def test_new_message(self, evt):
try:
msg = messaging.new_message(evt)
except capnp.lib.capnp.KjException:
msg = messaging.new_message(evt, random.randrange(200))
assert (time.monotonic() - msg.logMonoTime) < 0.1
assert not msg.valid
assert evt == msg.which()
@parameterized.expand(events)
def test_pub_sock(self, evt):
messaging.pub_sock(evt)
@parameterized.expand(events)
def test_sub_sock(self, evt):
messaging.sub_sock(evt)
@parameterized.expand([
(messaging.drain_sock, capnp._DynamicStructReader),
(messaging.drain_sock_raw, bytes),
])
def test_drain_sock(self, func, expected_type):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=1000)
zmq_sleep()
# no wait and no msgs in queue
msgs = func(sub_sock)
assert isinstance(msgs, list)
assert len(msgs) == 0
# no wait but msgs are queued up
num_msgs = random.randrange(3, 10)
for _ in range(num_msgs):
pub_sock.send(messaging.new_message(sock).to_bytes())
time.sleep(0.1)
msgs = func(sub_sock)
assert isinstance(msgs, list)
assert all(isinstance(msg, expected_type) for msg in msgs)
assert len(msgs) == num_msgs
def test_recv_sock(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=100)
zmq_sleep()
# no wait and no msg in queue, socket should timeout
recvd = messaging.recv_sock(sub_sock)
assert recvd is None
# no wait and one msg in queue
msg = random_carstate()
pub_sock.send(msg.to_bytes())
time.sleep(0.01)
recvd = messaging.recv_sock(sub_sock)
assert isinstance(recvd, capnp._DynamicStructReader)
# https://github.com/python/mypy/issues/13038
assert_carstate(msg.carState, recvd.carState)
def test_recv_one(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=1000)
zmq_sleep()
# no msg in queue, socket should timeout
recvd = messaging.recv_one(sub_sock)
assert recvd is None
# one msg in queue
msg = random_carstate()
pub_sock.send(msg.to_bytes())
recvd = messaging.recv_one(sub_sock)
assert isinstance(recvd, capnp._DynamicStructReader)
assert_carstate(msg.carState, recvd.carState)
@pytest.mark.xfail(condition="ZMQ" in os.environ, reason='ZMQ detected')
def test_recv_one_or_none(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock)
zmq_sleep()
# no msg in queue, socket shouldn't block
recvd = messaging.recv_one_or_none(sub_sock)
assert recvd is None
# one msg in queue
msg = random_carstate()
pub_sock.send(msg.to_bytes())
recvd = messaging.recv_one_or_none(sub_sock)
assert isinstance(recvd, capnp._DynamicStructReader)
assert_carstate(msg.carState, recvd.carState)
def test_recv_one_retry(self):
sock = "carState"
sock_timeout = 0.1
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=round(sock_timeout*1000))
zmq_sleep()
# this test doesn't work with ZMQ since multiprocessing interrupts it
if "ZMQ" not in os.environ:
# wait 5 socket timeouts and make sure it's still retrying
p = multiprocessing.Process(target=messaging.recv_one_retry, args=(sub_sock,))
p.start()
time.sleep(sock_timeout*5)
assert p.is_alive()
p.terminate()
# wait 5 socket timeouts before sending
msg = random_carstate()
start_time = time.monotonic()
delayed_send(sock_timeout*5, pub_sock, msg.to_bytes())
recvd = messaging.recv_one_retry(sub_sock)
assert (time.monotonic() - start_time) >= sock_timeout*5
assert isinstance(recvd, capnp._DynamicStructReader)
assert_carstate(msg.carState, recvd.carState)

View File

@ -1,160 +0,0 @@
import random
import time
from typing import Sized, cast
import cereal.messaging as messaging
from cereal.messaging.tests.test_messaging import events, random_sock, random_socks, \
random_bytes, random_carstate, assert_carstate, \
zmq_sleep
from cereal.services import SERVICE_LIST
class TestSubMaster:
def setup_method(self):
# ZMQ pub socket takes too long to die
# sleep to prevent multiple publishers error between tests
zmq_sleep(3)
def test_init(self):
sm = messaging.SubMaster(events)
for p in [sm.updated, sm.recv_time, sm.recv_frame, sm.alive,
sm.sock, sm.data, sm.logMonoTime, sm.valid]:
assert len(cast(Sized, p)) == len(events)
def test_init_state(self):
socks = random_socks()
sm = messaging.SubMaster(socks)
assert sm.frame == -1
assert not any(sm.updated.values())
assert not any(sm.seen.values())
on_demand = {s: SERVICE_LIST[s].frequency <= 1e-5 for s in sm.services}
assert all(sm.alive[s] == sm.valid[s] == sm.freq_ok[s] == on_demand[s] for s in sm.services)
assert all(t == 0. for t in sm.recv_time.values())
assert all(f == 0 for f in sm.recv_frame.values())
assert all(t == 0 for t in sm.logMonoTime.values())
for p in [sm.updated, sm.recv_time, sm.recv_frame, sm.alive,
sm.sock, sm.data, sm.logMonoTime, sm.valid]:
assert len(cast(Sized, p)) == len(socks)
def test_getitem(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sm = messaging.SubMaster([sock,])
zmq_sleep()
msg = random_carstate()
pub_sock.send(msg.to_bytes())
sm.update(1000)
assert_carstate(msg.carState, sm[sock])
# TODO: break this test up to individually test SubMaster.update and SubMaster.update_msgs
def test_update(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sm = messaging.SubMaster([sock,])
zmq_sleep()
for i in range(10):
msg = messaging.new_message(sock)
pub_sock.send(msg.to_bytes())
sm.update(1000)
assert sm.frame == i
assert all(sm.updated.values())
def test_update_timeout(self):
sock = random_sock()
sm = messaging.SubMaster([sock,])
timeout = random.randrange(1000, 3000)
start_time = time.monotonic()
sm.update(timeout)
t = time.monotonic() - start_time
assert t >= timeout/1000.
assert t < 3
assert not any(sm.updated.values())
def test_avg_frequency_checks(self):
for poll in (True, False):
sm = messaging.SubMaster(["modelV2", "carParams", "carState", "cameraOdometry", "liveCalibration"],
poll=("modelV2" if poll else None),
frequency=(20. if not poll else None))
checks = {
"carState": (20, 20),
"modelV2": (20, 20 if poll else 10),
"cameraOdometry": (20, 10),
"liveCalibration": (4, 4),
"carParams": (None, None),
"userBookmark": (None, None),
}
for service, (max_freq, min_freq) in checks.items():
if max_freq is not None:
assert sm._check_avg_freq(service)
assert sm.freq_tracker[service].max_freq == max_freq*1.2
assert sm.freq_tracker[service].min_freq == min_freq*0.8
else:
assert not sm._check_avg_freq(service)
def test_alive(self):
pass
def test_ignore_alive(self):
pass
def test_valid(self):
pass
# SubMaster should always conflate
def test_conflate(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sm = messaging.SubMaster([sock,])
n = 10
for i in range(n+1):
msg = messaging.new_message(sock)
msg.carState.vEgo = i
pub_sock.send(msg.to_bytes())
time.sleep(0.01)
sm.update(1000)
assert sm[sock].vEgo == n
class TestPubMaster:
def setup_method(self):
# ZMQ pub socket takes too long to die
# sleep to prevent multiple publishers error between tests
zmq_sleep(3)
def test_init(self):
messaging.PubMaster(events)
def test_send(self):
socks = random_socks()
pm = messaging.PubMaster(socks)
sub_socks = {s: messaging.sub_sock(s, conflate=True, timeout=1000) for s in socks}
zmq_sleep()
# PubMaster accepts either a capnp msg builder or bytes
for capnp in [True, False]:
for i in range(100):
sock = socks[i % len(socks)]
if capnp:
try:
msg = messaging.new_message(sock)
except Exception:
msg = messaging.new_message(sock, random.randrange(50))
else:
msg = random_bytes()
pm.send(sock, msg)
recvd = sub_socks[sock].receive()
if capnp:
msg.clear_write_flag()
msg = msg.to_bytes()
assert msg == recvd, i

View File

@ -1,21 +0,0 @@
import os
import tempfile
from typing import Dict
from parameterized import parameterized
import cereal.services as services
from cereal.services import SERVICE_LIST
class TestServices:
@parameterized.expand(SERVICE_LIST.keys())
def test_services(self, s):
service = SERVICE_LIST[s]
assert service.frequency <= 104
assert service.decimation != 0
def test_generated_header(self):
with tempfile.NamedTemporaryFile(suffix=".h") as f:
ret = os.system(f"python3 {services.__file__} > {f.name} && clang++ {f.name} -std=c++11")
assert ret == 0, "generated services header is not valid C"

98
cereal/services.h Normal file
View File

@ -0,0 +1,98 @@
/* THIS IS AN AUTOGENERATED FILE, PLEASE EDIT services.py */
#ifndef __SERVICES_H
#define __SERVICES_H
#include <map>
#include <string>
struct service { std::string name; bool should_log; int frequency; int decimation; };
static std::map<std::string, service> services = {
{ "gyroscope", {"gyroscope", true, 104, 104}},
{ "gyroscope2", {"gyroscope2", true, 100, 100}},
{ "accelerometer", {"accelerometer", true, 104, 104}},
{ "accelerometer2", {"accelerometer2", true, 100, 100}},
{ "magnetometer", {"magnetometer", true, 25, 25}},
{ "lightSensor", {"lightSensor", true, 100, 100}},
{ "temperatureSensor", {"temperatureSensor", true, 2, 200}},
{ "temperatureSensor2", {"temperatureSensor2", true, 2, 200}},
{ "gpsNMEA", {"gpsNMEA", true, 9, -1}},
{ "deviceState", {"deviceState", true, 2, 1}},
{ "can", {"can", true, 100, 1223}},
{ "controlsState", {"controlsState", true, 100, 10}},
{ "pandaStates", {"pandaStates", true, 10, 1}},
{ "peripheralState", {"peripheralState", true, 2, 1}},
{ "radarState", {"radarState", true, 20, 5}},
{ "roadEncodeIdx", {"roadEncodeIdx", false, 20, 1}},
{ "liveTracks", {"liveTracks", true, 20, -1}},
{ "sendcan", {"sendcan", true, 100, 139}},
{ "logMessage", {"logMessage", true, 0, -1}},
{ "errorLogMessage", {"errorLogMessage", true, 0, 1}},
{ "liveCalibration", {"liveCalibration", true, 4, 4}},
{ "liveTorqueParameters", {"liveTorqueParameters", true, 4, 1}},
{ "liveDelay", {"liveDelay", true, 4, 1}},
{ "androidLog", {"androidLog", true, 0, -1}},
{ "carState", {"carState", true, 100, 10}},
{ "carControl", {"carControl", true, 100, 10}},
{ "carOutput", {"carOutput", true, 100, 10}},
{ "longitudinalPlan", {"longitudinalPlan", true, 20, 5}},
{ "procLog", {"procLog", true, 0, 15}},
{ "gpsLocationExternal", {"gpsLocationExternal", true, 10, 10}},
{ "gpsLocation", {"gpsLocation", true, 1, 1}},
{ "ubloxGnss", {"ubloxGnss", true, 10, -1}},
{ "qcomGnss", {"qcomGnss", true, 2, -1}},
{ "gnssMeasurements", {"gnssMeasurements", true, 10, 10}},
{ "clocks", {"clocks", true, 0, 1}},
{ "ubloxRaw", {"ubloxRaw", true, 20, -1}},
{ "livePose", {"livePose", true, 20, 4}},
{ "liveLocationKalman", {"liveLocationKalman", true, 20, -1}},
{ "liveParameters", {"liveParameters", true, 20, 5}},
{ "cameraOdometry", {"cameraOdometry", true, 20, 5}},
{ "thumbnail", {"thumbnail", true, 0, 1}},
{ "onroadEvents", {"onroadEvents", true, 1, 1}},
{ "carParams", {"carParams", true, 0, 1}},
{ "roadCameraState", {"roadCameraState", true, 20, 20}},
{ "driverCameraState", {"driverCameraState", true, 20, 20}},
{ "driverEncodeIdx", {"driverEncodeIdx", false, 20, 1}},
{ "driverStateV2", {"driverStateV2", true, 20, 10}},
{ "driverMonitoringState", {"driverMonitoringState", true, 20, 10}},
{ "wideRoadEncodeIdx", {"wideRoadEncodeIdx", false, 20, 1}},
{ "wideRoadCameraState", {"wideRoadCameraState", true, 20, 20}},
{ "drivingModelData", {"drivingModelData", true, 20, 10}},
{ "modelV2", {"modelV2", true, 20, 40}},
{ "managerState", {"managerState", true, 2, 1}},
{ "uploaderState", {"uploaderState", true, 0, 1}},
{ "navInstruction", {"navInstruction", true, 1, 10}},
{ "navRoute", {"navRoute", true, 0, -1}},
{ "navThumbnail", {"navThumbnail", true, 0, -1}},
{ "navModel", {"navModel", true, 2, 4}},
{ "mapRenderState", {"mapRenderState", true, 2, 1}},
{ "uiPlan", {"uiPlan", true, 20, 40}},
{ "qRoadEncodeIdx", {"qRoadEncodeIdx", false, 20, -1}},
{ "userFlag", {"userFlag", true, 0, 1}},
{ "microphone", {"microphone", true, 10, 10}},
{ "uiDebug", {"uiDebug", true, 0, 1}},
{ "testJoystick", {"testJoystick", true, 0, -1}},
{ "roadEncodeData", {"roadEncodeData", false, 20, -1}},
{ "driverEncodeData", {"driverEncodeData", false, 20, -1}},
{ "wideRoadEncodeData", {"wideRoadEncodeData", false, 20, -1}},
{ "qRoadEncodeData", {"qRoadEncodeData", false, 20, -1}},
{ "livestreamWideRoadEncodeIdx", {"livestreamWideRoadEncodeIdx", false, 20, -1}},
{ "livestreamRoadEncodeIdx", {"livestreamRoadEncodeIdx", false, 20, -1}},
{ "livestreamDriverEncodeIdx", {"livestreamDriverEncodeIdx", false, 20, -1}},
{ "livestreamWideRoadEncodeData", {"livestreamWideRoadEncodeData", false, 20, -1}},
{ "livestreamRoadEncodeData", {"livestreamRoadEncodeData", false, 20, -1}},
{ "livestreamDriverEncodeData", {"livestreamDriverEncodeData", false, 20, -1}},
{ "customReservedRawData0", {"customReservedRawData0", true, 0, -1}},
{ "customReservedRawData1", {"customReservedRawData1", true, 0, -1}},
{ "customReservedRawData2", {"customReservedRawData2", true, 0, -1}},
{ "frogpilotCarControl", {"frogpilotCarControl", true, 100, 10}},
{ "frogpilotCarParams", {"frogpilotCarParams", true, 0, 1}},
{ "frogpilotCarState", {"frogpilotCarState", true, 100, 10}},
{ "frogpilotControlsState", {"frogpilotControlsState", true, 100, 10}},
{ "frogpilotDeviceState", {"frogpilotDeviceState", true, 2, 1}},
{ "frogpilotModelV2", {"frogpilotModelV2", true, 20, 40}},
{ "frogpilotNavigation", {"frogpilotNavigation", true, 1, 10}},
{ "frogpilotOnroadEvents", {"frogpilotOnroadEvents", true, 1, 1}},
{ "frogpilotPlan", {"frogpilotPlan", true, 20, 5}},
{ "frogpilotRadarState", {"frogpilotRadarState", true, 20, 5}},
};
#endif

View File

@ -16,16 +16,14 @@ _services: dict[str, tuple] = {
"gyroscope2": (True, 100., 100),
"accelerometer": (True, 104., 104),
"accelerometer2": (True, 100., 100),
"magnetometer": (True, 25.),
"magnetometer": (True, 25., 25),
"lightSensor": (True, 100., 100),
"temperatureSensor": (True, 2., 200),
"temperatureSensor2": (True, 2., 200),
"gpsNMEA": (True, 9.),
"deviceState": (True, 2., 1),
"touch": (True, 20., 1),
"can": (True, 100., 2053), # decimation gives ~3 msgs in a full segment
"can": (True, 100., 1223), # decimation gives ~5 msgs in a full segment
"controlsState": (True, 100., 10),
"selfdriveState": (True, 100., 10),
"pandaStates": (True, 10., 1),
"peripheralState": (True, 2., 1),
"radarState": (True, 20., 5),
@ -41,8 +39,7 @@ _services: dict[str, tuple] = {
"carState": (True, 100., 10),
"carControl": (True, 100., 10),
"carOutput": (True, 100., 10),
"longitudinalPlan": (True, 20., 10),
"driverAssistance": (True, 20., 20),
"longitudinalPlan": (True, 20., 5),
"procLog": (True, 0.5, 15),
"gpsLocationExternal": (True, 10., 10),
"gpsLocation": (True, 1., 1),
@ -52,9 +49,10 @@ _services: dict[str, tuple] = {
"clocks": (True, 0.1, 1),
"ubloxRaw": (True, 20.),
"livePose": (True, 20., 4),
"liveLocationKalman": (True, 20.),
"liveParameters": (True, 20., 5),
"cameraOdometry": (True, 20., 10),
"thumbnail": (True, 1 / 60., 1),
"cameraOdometry": (True, 20., 5),
"thumbnail": (True, 0.2, 1),
"onroadEvents": (True, 1., 1),
"carParams": (True, 0.02, 1),
"roadCameraState": (True, 20., 20),
@ -65,23 +63,22 @@ _services: dict[str, tuple] = {
"wideRoadEncodeIdx": (False, 20., 1),
"wideRoadCameraState": (True, 20., 20),
"drivingModelData": (True, 20., 10),
"modelV2": (True, 20.),
"modelV2": (True, 20., 40),
"managerState": (True, 2., 1),
"uploaderState": (True, 0., 1),
"navInstruction": (True, 1., 10),
"navRoute": (True, 0.),
"navThumbnail": (True, 0.),
"navModel": (True, 2., 4.),
"mapRenderState": (True, 2., 1.),
"uiPlan": (True, 20., 40.),
"qRoadEncodeIdx": (False, 20.),
"userBookmark": (True, 0., 1),
"soundPressure": (True, 10., 10),
"rawAudioData": (False, 20.),
"bookmarkButton": (True, 0., 1),
"audioFeedback": (True, 0., 1),
"userFlag": (True, 0., 1),
"microphone": (True, 10., 10),
# debug
"uiDebug": (True, 0., 1),
"testJoystick": (True, 0.),
"alertDebug": (True, 20., 5),
"roadEncodeData": (False, 20.),
"driverEncodeData": (False, 20.),
"wideRoadEncodeData": (False, 20.),
@ -95,8 +92,18 @@ _services: dict[str, tuple] = {
"customReservedRawData0": (True, 0.),
"customReservedRawData1": (True, 0.),
"customReservedRawData2": (True, 0.),
"dpControlsState": (False, 100., 10),
"modelExt": (True, 20.),
# FrogPilot
"frogpilotCarControl": (True, 100., 10),
"frogpilotCarParams": (True, 0.02, 1),
"frogpilotCarState": (True, 100., 10),
"frogpilotControlsState": (True, 100., 10),
"frogpilotDeviceState": (True, 2., 1),
"frogpilotModelV2": (True, 20., 40),
"frogpilotNavigation": (True, 1., 10),
"frogpilotOnroadEvents": (True, 1., 1),
"frogpilotPlan": (True, 20., 5),
"frogpilotRadarState": (True, 20., 5),
}
SERVICE_LIST = {name: Service(*vals) for
idx, (name, vals) in enumerate(_services.items())}
@ -111,12 +118,12 @@ def build_header():
h += "#include <map>\n"
h += "#include <string>\n"
h += "struct service { std::string name; bool should_log; float frequency; int decimation; };\n"
h += "struct service { std::string name; bool should_log; int frequency; int decimation; };\n"
h += "static std::map<std::string, service> services = {\n"
for k, v in SERVICE_LIST.items():
should_log = "true" if v.should_log else "false"
decimation = -1 if v.decimation is None else v.decimation
h += ' { "%s", {"%s", %s, %f, %d}},\n' % \
h += ' { "%s", {"%s", %s, %d, %d}},\n' % \
(k, k, should_log, v.frequency, decimation)
h += "};\n"

View File

@ -1,29 +0,0 @@
Import('env', 'envCython', 'arch')
common_libs = [
'params.cc',
'swaglog.cc',
'util.cc',
'ratekeeper.cc',
'clutil.cc',
]
_common = env.Library('common', common_libs, LIBS="json11")
Export('_common')
if GetOption('extras'):
env.Program('tests/test_common',
['tests/test_runner.cc', 'tests/test_params.cc', 'tests/test_util.cc', 'tests/test_swaglog.cc'],
LIBS=[_common, 'json11', 'zmq', 'pthread'])
# Cython bindings
params_python = envCython.Program('params_pyx.so', 'params_pyx.pyx', LIBS=envCython['LIBS'] + [_common, 'zmq', 'json11'])
SConscript([
'transformations/SConscript',
])
Import('transformations_python')
common_python = [params_python, transformations_python]
Export('common_python')

View File

@ -1,48 +0,0 @@
import jwt
import os
import requests
from datetime import datetime, timedelta, UTC
from openpilot.system.hardware.hw import Paths
from openpilot.system.version import get_version
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
class Api:
def __init__(self, dongle_id):
self.dongle_id = dongle_id
with open(Paths.persist_root()+'/comma/id_rsa') as f:
self.private_key = f.read()
def get(self, *args, **kwargs):
return self.request('GET', *args, **kwargs)
def post(self, *args, **kwargs):
return self.request('POST', *args, **kwargs)
def request(self, method, endpoint, timeout=None, access_token=None, **params):
return api_get(endpoint, method=method, timeout=timeout, access_token=access_token, **params)
def get_token(self, payload_extra=None, expiry_hours=1):
now = datetime.now(UTC).replace(tzinfo=None)
payload = {
'identity': self.dongle_id,
'nbf': now,
'iat': now,
'exp': now + timedelta(hours=expiry_hours)
}
if payload_extra is not None:
payload.update(payload_extra)
token = jwt.encode(payload, self.private_key, algorithm='RS256')
if isinstance(token, bytes):
token = token.decode('utf8')
return token
def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
headers['User-Agent'] = "openpilot-" + get_version()
return requests.request(method, API_HOST + "/" + endpoint, timeout=timeout, headers=headers, params=params)

49
common/api/__init__.py Normal file
View File

@ -0,0 +1,49 @@
import jwt
import os
import requests
from datetime import datetime, timedelta
from openpilot.system.hardware.hw import Paths
from openpilot.system.version import get_version
from openpilot.frogpilot.common.frogpilot_utilities import use_konik_server
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
KONIK_API_HOST = os.getenv('API_HOST', 'https://api.konik.ai')
class Api:
def __init__(self, dongle_id):
self.dongle_id = dongle_id
with open(Paths.persist_root()+'/comma/id_rsa') as f:
self.private_key = f.read()
def get(self, *args, **kwargs):
return self.request('GET', *args, **kwargs)
def post(self, *args, **kwargs):
return self.request('POST', *args, **kwargs)
def request(self, method, endpoint, timeout=None, access_token=None, **params):
return api_get(endpoint, method=method, timeout=timeout, access_token=access_token, **params)
def get_token(self, expiry_hours=1):
now = datetime.utcnow()
payload = {
'identity': self.dongle_id,
'nbf': now,
'iat': now,
'exp': now + timedelta(hours=expiry_hours)
}
token = jwt.encode(payload, self.private_key, algorithm='RS256')
if isinstance(token, bytes):
token = token.decode('utf8')
return token
def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
headers['User-Agent'] = "openpilot-" + get_version()
return requests.request(method, (KONIK_API_HOST if use_konik_server() else API_HOST) + "/" + endpoint, timeout=timeout, headers=headers, params=params)

View File

@ -96,3 +96,109 @@ cl_program cl_program_from_source(cl_context ctx, cl_device_id device_id, const
}
return prg;
}
cl_program cl_program_from_binary(cl_context ctx, cl_device_id device_id, const uint8_t* binary, size_t length, const char* args) {
cl_program prg = CL_CHECK_ERR(clCreateProgramWithBinary(ctx, 1, &device_id, &length, &binary, NULL, &err));
if (int err = clBuildProgram(prg, 1, &device_id, args, NULL, NULL); err != 0) {
cl_print_build_errors(prg, device_id);
assert(0);
}
return prg;
}
// Given a cl code and return a string representation
#define CL_ERR_TO_STR(err) case err: return #err
const char* cl_get_error_string(int err) {
switch (err) {
CL_ERR_TO_STR(CL_SUCCESS);
CL_ERR_TO_STR(CL_DEVICE_NOT_FOUND);
CL_ERR_TO_STR(CL_DEVICE_NOT_AVAILABLE);
CL_ERR_TO_STR(CL_COMPILER_NOT_AVAILABLE);
CL_ERR_TO_STR(CL_MEM_OBJECT_ALLOCATION_FAILURE);
CL_ERR_TO_STR(CL_OUT_OF_RESOURCES);
CL_ERR_TO_STR(CL_OUT_OF_HOST_MEMORY);
CL_ERR_TO_STR(CL_PROFILING_INFO_NOT_AVAILABLE);
CL_ERR_TO_STR(CL_MEM_COPY_OVERLAP);
CL_ERR_TO_STR(CL_IMAGE_FORMAT_MISMATCH);
CL_ERR_TO_STR(CL_IMAGE_FORMAT_NOT_SUPPORTED);
CL_ERR_TO_STR(CL_MAP_FAILURE);
CL_ERR_TO_STR(CL_MISALIGNED_SUB_BUFFER_OFFSET);
CL_ERR_TO_STR(CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST);
CL_ERR_TO_STR(CL_COMPILE_PROGRAM_FAILURE);
CL_ERR_TO_STR(CL_LINKER_NOT_AVAILABLE);
CL_ERR_TO_STR(CL_LINK_PROGRAM_FAILURE);
CL_ERR_TO_STR(CL_DEVICE_PARTITION_FAILED);
CL_ERR_TO_STR(CL_KERNEL_ARG_INFO_NOT_AVAILABLE);
CL_ERR_TO_STR(CL_INVALID_VALUE);
CL_ERR_TO_STR(CL_INVALID_DEVICE_TYPE);
CL_ERR_TO_STR(CL_INVALID_PLATFORM);
CL_ERR_TO_STR(CL_INVALID_DEVICE);
CL_ERR_TO_STR(CL_INVALID_CONTEXT);
CL_ERR_TO_STR(CL_INVALID_QUEUE_PROPERTIES);
CL_ERR_TO_STR(CL_INVALID_COMMAND_QUEUE);
CL_ERR_TO_STR(CL_INVALID_HOST_PTR);
CL_ERR_TO_STR(CL_INVALID_MEM_OBJECT);
CL_ERR_TO_STR(CL_INVALID_IMAGE_FORMAT_DESCRIPTOR);
CL_ERR_TO_STR(CL_INVALID_IMAGE_SIZE);
CL_ERR_TO_STR(CL_INVALID_SAMPLER);
CL_ERR_TO_STR(CL_INVALID_BINARY);
CL_ERR_TO_STR(CL_INVALID_BUILD_OPTIONS);
CL_ERR_TO_STR(CL_INVALID_PROGRAM);
CL_ERR_TO_STR(CL_INVALID_PROGRAM_EXECUTABLE);
CL_ERR_TO_STR(CL_INVALID_KERNEL_NAME);
CL_ERR_TO_STR(CL_INVALID_KERNEL_DEFINITION);
CL_ERR_TO_STR(CL_INVALID_KERNEL);
CL_ERR_TO_STR(CL_INVALID_ARG_INDEX);
CL_ERR_TO_STR(CL_INVALID_ARG_VALUE);
CL_ERR_TO_STR(CL_INVALID_ARG_SIZE);
CL_ERR_TO_STR(CL_INVALID_KERNEL_ARGS);
CL_ERR_TO_STR(CL_INVALID_WORK_DIMENSION);
CL_ERR_TO_STR(CL_INVALID_WORK_GROUP_SIZE);
CL_ERR_TO_STR(CL_INVALID_WORK_ITEM_SIZE);
CL_ERR_TO_STR(CL_INVALID_GLOBAL_OFFSET);
CL_ERR_TO_STR(CL_INVALID_EVENT_WAIT_LIST);
CL_ERR_TO_STR(CL_INVALID_EVENT);
CL_ERR_TO_STR(CL_INVALID_OPERATION);
CL_ERR_TO_STR(CL_INVALID_GL_OBJECT);
CL_ERR_TO_STR(CL_INVALID_BUFFER_SIZE);
CL_ERR_TO_STR(CL_INVALID_MIP_LEVEL);
CL_ERR_TO_STR(CL_INVALID_GLOBAL_WORK_SIZE);
CL_ERR_TO_STR(CL_INVALID_PROPERTY);
CL_ERR_TO_STR(CL_INVALID_IMAGE_DESCRIPTOR);
CL_ERR_TO_STR(CL_INVALID_COMPILER_OPTIONS);
CL_ERR_TO_STR(CL_INVALID_LINKER_OPTIONS);
CL_ERR_TO_STR(CL_INVALID_DEVICE_PARTITION_COUNT);
case -69: return "CL_INVALID_PIPE_SIZE";
case -70: return "CL_INVALID_DEVICE_QUEUE";
case -71: return "CL_INVALID_SPEC_ID";
case -72: return "CL_MAX_SIZE_RESTRICTION_EXCEEDED";
case -1002: return "CL_INVALID_D3D10_DEVICE_KHR";
case -1003: return "CL_INVALID_D3D10_RESOURCE_KHR";
case -1004: return "CL_D3D10_RESOURCE_ALREADY_ACQUIRED_KHR";
case -1005: return "CL_D3D10_RESOURCE_NOT_ACQUIRED_KHR";
case -1006: return "CL_INVALID_D3D11_DEVICE_KHR";
case -1007: return "CL_INVALID_D3D11_RESOURCE_KHR";
case -1008: return "CL_D3D11_RESOURCE_ALREADY_ACQUIRED_KHR";
case -1009: return "CL_D3D11_RESOURCE_NOT_ACQUIRED_KHR";
case -1010: return "CL_INVALID_DX9_MEDIA_ADAPTER_KHR";
case -1011: return "CL_INVALID_DX9_MEDIA_SURFACE_KHR";
case -1012: return "CL_DX9_MEDIA_SURFACE_ALREADY_ACQUIRED_KHR";
case -1013: return "CL_DX9_MEDIA_SURFACE_NOT_ACQUIRED_KHR";
case -1093: return "CL_INVALID_EGL_OBJECT_KHR";
case -1092: return "CL_EGL_RESOURCE_NOT_ACQUIRED_KHR";
case -1001: return "CL_PLATFORM_NOT_FOUND_KHR";
case -1057: return "CL_DEVICE_PARTITION_FAILED_EXT";
case -1058: return "CL_INVALID_PARTITION_COUNT_EXT";
case -1059: return "CL_INVALID_PARTITION_NAME_EXT";
case -1094: return "CL_INVALID_ACCELERATOR_INTEL";
case -1095: return "CL_INVALID_ACCELERATOR_TYPE_INTEL";
case -1096: return "CL_INVALID_ACCELERATOR_DESCRIPTOR_INTEL";
case -1097: return "CL_ACCELERATOR_TYPE_NOT_SUPPORTED_INTEL";
case -1000: return "CL_INVALID_GL_SHAREGROUP_REFERENCE_KHR";
case -1098: return "CL_INVALID_VA_API_MEDIA_ADAPTER_INTEL";
case -1099: return "CL_INVALID_VA_API_MEDIA_SURFACE_INTEL";
case -1100: return "CL_VA_API_MEDIA_SURFACE_ALREADY_ACQUIRED_INTEL";
case -1101: return "CL_VA_API_MEDIA_SURFACE_NOT_ACQUIRED_INTEL";
default: return "CL_UNKNOWN_ERROR";
}
}

View File

@ -25,4 +25,6 @@ cl_device_id cl_get_device_id(cl_device_type device_type);
cl_context cl_create_context(cl_device_id device_id);
void cl_release_context(cl_context context);
cl_program cl_program_from_source(cl_context ctx, cl_device_id device_id, const std::string& src, const char* args = nullptr);
cl_program cl_program_from_binary(cl_context ctx, cl_device_id device_id, const uint8_t* binary, size_t length, const char* args = nullptr);
cl_program cl_program_from_file(cl_context ctx, cl_device_id device_id, const char* path, const char* args);
const char* cl_get_error_string(int err);

View File

@ -1,23 +0,0 @@
import numpy as np
# conversions
class CV:
# Speed
MPH_TO_KPH = 1.609344
KPH_TO_MPH = 1. / MPH_TO_KPH
MS_TO_KPH = 3.6
KPH_TO_MS = 1. / MS_TO_KPH
MS_TO_MPH = MS_TO_KPH * KPH_TO_MPH
MPH_TO_MS = MPH_TO_KPH * KPH_TO_MS
MS_TO_KNOTS = 1.9438
KNOTS_TO_MS = 1. / MS_TO_KNOTS
# Angle
DEG_TO_RAD = np.pi / 180.
RAD_TO_DEG = 1. / DEG_TO_RAD
# Mass
LB_TO_KG = 0.453592
ACCELERATION_DUE_TO_GRAVITY = 9.81 # m/s^2

27
common/conversions.py Normal file
View File

@ -0,0 +1,27 @@
import numpy as np
class Conversions:
# Speed
MPH_TO_KPH = 1.609344
KPH_TO_MPH = 1. / MPH_TO_KPH
MS_TO_KPH = 3.6
KPH_TO_MS = 1. / MS_TO_KPH
MS_TO_MPH = MS_TO_KPH * KPH_TO_MPH
MPH_TO_MS = MPH_TO_KPH * KPH_TO_MS
MS_TO_KNOTS = 1.9438
KNOTS_TO_MS = 1. / MS_TO_KNOTS
# Distance
METER_TO_FOOT = 3.28084
FOOT_TO_METER = 1. / METER_TO_FOOT
METER_TO_MILE = METER_TO_FOOT / 5280
MILE_TO_METER = 1. / METER_TO_MILE
CM_TO_INCH = 1. / 2.54
INCH_TO_CM = 1. / CM_TO_INCH
# Angle
DEG_TO_RAD = np.pi / 180.
RAD_TO_DEG = 1. / DEG_TO_RAD
# Mass
LB_TO_KG = 0.453592

8
common/ffi_wrapper.py Normal file
View File

@ -0,0 +1,8 @@
import platform
def suffix():
if platform.system() == "Darwin":
return ".dylib"
else:
return ".so"

View File

@ -1,10 +1,6 @@
import io
import os
import tempfile
import contextlib
import zstandard as zstd
LOG_COMPRESSION_LEVEL = 10 # little benefit up to level 15. level ~17 is a small step change
class CallbackReader:
@ -39,20 +35,3 @@ def atomic_write_in_dir(path: str, mode: str = 'w', buffering: int = -1, encodin
yield tmp_file
tmp_file_name = tmp_file.name
os.replace(tmp_file_name, path)
def get_upload_stream(filepath: str, should_compress: bool) -> tuple[io.BufferedIOBase, int]:
if not should_compress:
file_size = os.path.getsize(filepath)
file_stream = open(filepath, "rb")
return file_stream, file_size
# Compress the file on the fly
compressed_stream = io.BytesIO()
compressor = zstd.ZstdCompressor(level=LOG_COMPRESSION_LEVEL)
with open(filepath, "rb") as f:
compressor.copy_stream(f, compressed_stream)
compressed_size = compressed_stream.tell()
compressed_stream.seek(0)
return compressed_stream, compressed_size

View File

@ -1,4 +1,5 @@
class FirstOrderFilter:
# first order filter
def __init__(self, x0, rc, dt, initialized=True):
self.x = x0
self.dt = dt

84
common/gpio.cc Normal file
View File

@ -0,0 +1,84 @@
#include "common/gpio.h"
#include <string>
#ifdef __APPLE__
int gpio_init(int pin_nr, bool output) {
return 0;
}
int gpio_set(int pin_nr, bool high) {
return 0;
}
int gpiochip_get_ro_value_fd(const char* consumer_label, int gpiochiop_id, int pin_nr) {
return 0;
}
#else
#include <fcntl.h>
#include <unistd.h>
#include <cstring>
#include <linux/gpio.h>
#include <sys/ioctl.h>
#include "common/util.h"
#include "common/swaglog.h"
int gpio_init(int pin_nr, bool output) {
char pin_dir_path[50];
int pin_dir_path_len = snprintf(pin_dir_path, sizeof(pin_dir_path),
"/sys/class/gpio/gpio%d/direction", pin_nr);
if (pin_dir_path_len <= 0) {
return -1;
}
const char *value = output ? "out" : "in";
return util::write_file(pin_dir_path, (void*)value, strlen(value));
}
int gpio_set(int pin_nr, bool high) {
char pin_val_path[50];
int pin_val_path_len = snprintf(pin_val_path, sizeof(pin_val_path),
"/sys/class/gpio/gpio%d/value", pin_nr);
if (pin_val_path_len <= 0) {
return -1;
}
return util::write_file(pin_val_path, (void*)(high ? "1" : "0"), 1);
}
int gpiochip_get_ro_value_fd(const char* consumer_label, int gpiochiop_id, int pin_nr) {
// Assumed that all interrupt pins are unexported and rights are given to
// read from gpiochip0.
std::string gpiochip_path = "/dev/gpiochip" + std::to_string(gpiochiop_id);
int fd = open(gpiochip_path.c_str(), O_RDONLY);
if (fd < 0) {
LOGE("Error opening gpiochip0 fd");
return -1;
}
// Setup event
struct gpioevent_request rq;
rq.lineoffset = pin_nr;
rq.handleflags = GPIOHANDLE_REQUEST_INPUT;
/* Requesting both edges as the data ready pulse from the lsm6ds sensor is
very short(75us) and is mostly detected as falling edge instead of rising.
So if it is detected as rising the following falling edge is skipped. */
rq.eventflags = GPIOEVENT_REQUEST_BOTH_EDGES;
strncpy(rq.consumer_label, consumer_label, std::size(rq.consumer_label) - 1);
int ret = util::safe_ioctl(fd, GPIO_GET_LINEEVENT_IOCTL, &rq);
if (ret == -1) {
LOGE("Unable to get line event from ioctl : %s", strerror(errno));
close(fd);
return -1;
}
close(fd);
return rq.fd;
}
#endif

33
common/gpio.h Normal file
View File

@ -0,0 +1,33 @@
#pragma once
// Pin definitions
#ifdef QCOM2
#define GPIO_HUB_RST_N 30
#define GPIO_UBLOX_RST_N 32
#define GPIO_UBLOX_SAFEBOOT_N 33
#define GPIO_GNSS_PWR_EN 34 /* SCHEMATIC LABEL: GPIO_UBLOX_PWR_EN */
#define GPIO_STM_RST_N 124
#define GPIO_STM_BOOT0 134
#define GPIO_BMX_ACCEL_INT 21
#define GPIO_BMX_GYRO_INT 23
#define GPIO_BMX_MAGN_INT 87
#define GPIO_LSM_INT 84
#define GPIOCHIP_INT 0
#else
#define GPIO_HUB_RST_N 0
#define GPIO_UBLOX_RST_N 0
#define GPIO_UBLOX_SAFEBOOT_N 0
#define GPIO_GNSS_PWR_EN 0 /* SCHEMATIC LABEL: GPIO_UBLOX_PWR_EN */
#define GPIO_STM_RST_N 0
#define GPIO_STM_BOOT0 0
#define GPIO_BMX_ACCEL_INT 0
#define GPIO_BMX_GYRO_INT 0
#define GPIO_BMX_MAGN_INT 0
#define GPIO_LSM_INT 0
#define GPIOCHIP_INT 0
#endif
int gpio_init(int pin_nr, bool output);
int gpio_set(int pin_nr, bool high);
int gpiochip_get_ro_value_fd(const char* consumer_label, int gpiochiop_id, int pin_nr);

View File

@ -1,6 +1,4 @@
import os
import fcntl
import ctypes
from functools import cache
def gpio_init(pin: int, output: bool) -> None:
@ -54,36 +52,3 @@ def get_irqs_for_action(action: str) -> list[str]:
if irq.isdigit() and action in get_irq_action(irq):
ret.append(irq)
return ret
# *** gpiochip ***
class gpioevent_data(ctypes.Structure):
_fields_ = [
("timestamp", ctypes.c_uint64),
("id", ctypes.c_uint32),
]
class gpioevent_request(ctypes.Structure):
_fields_ = [
("lineoffset", ctypes.c_uint32),
("handleflags", ctypes.c_uint32),
("eventflags", ctypes.c_uint32),
("label", ctypes.c_char * 32),
("fd", ctypes.c_int)
]
def gpiochip_get_ro_value_fd(label: str, gpiochip_id: int, pin: int) -> int:
GPIOEVENT_REQUEST_BOTH_EDGES = 0x3
GPIOHANDLE_REQUEST_INPUT = 0x1
GPIO_GET_LINEEVENT_IOCTL = 0xc030b404
rq = gpioevent_request()
rq.lineoffset = pin
rq.handleflags = GPIOHANDLE_REQUEST_INPUT
rq.eventflags = GPIOEVENT_REQUEST_BOTH_EDGES
rq.label = label.encode('utf-8')[:31] + b'\0'
fd = os.open(f"/dev/gpiochip{gpiochip_id}", os.O_RDONLY)
fcntl.ioctl(fd, GPIO_GET_LINEEVENT_IOCTL, rq)
os.close(fd)
return int(rq.fd)

View File

@ -1,8 +0,0 @@
from openpilot.common.params import Params
def get_gps_location_service(params: Params) -> str:
if params.get_bool("UbloxAvailable"):
return "gpsLocationExternal"
else:
return "gpsLocation"

92
common/i2c.cc Normal file
View File

@ -0,0 +1,92 @@
#include "common/i2c.h"
#include <fcntl.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <cassert>
#include <cstdio>
#include <stdexcept>
#include "common/swaglog.h"
#include "common/util.h"
#define UNUSED(x) (void)(x)
#ifdef QCOM2
// TODO: decide if we want to install libi2c-dev everywhere
extern "C" {
#include <linux/i2c-dev.h>
#include <i2c/smbus.h>
}
I2CBus::I2CBus(uint8_t bus_id) {
char bus_name[20];
snprintf(bus_name, 20, "/dev/i2c-%d", bus_id);
i2c_fd = HANDLE_EINTR(open(bus_name, O_RDWR));
if (i2c_fd < 0) {
throw std::runtime_error("Failed to open I2C bus");
}
}
I2CBus::~I2CBus() {
if (i2c_fd >= 0) {
close(i2c_fd);
}
}
int I2CBus::read_register(uint8_t device_address, uint register_address, uint8_t *buffer, uint8_t len) {
std::lock_guard lk(m);
int ret = 0;
ret = HANDLE_EINTR(ioctl(i2c_fd, I2C_SLAVE, device_address));
if (ret < 0) { goto fail; }
ret = i2c_smbus_read_i2c_block_data(i2c_fd, register_address, len, buffer);
if ((ret < 0) || (ret != len)) { goto fail; }
fail:
return ret;
}
int I2CBus::set_register(uint8_t device_address, uint register_address, uint8_t data) {
std::lock_guard lk(m);
int ret = 0;
ret = HANDLE_EINTR(ioctl(i2c_fd, I2C_SLAVE, device_address));
if (ret < 0) { goto fail; }
ret = i2c_smbus_write_byte_data(i2c_fd, register_address, data);
if (ret < 0) { goto fail; }
fail:
return ret;
}
#else
I2CBus::I2CBus(uint8_t bus_id) {
UNUSED(bus_id);
i2c_fd = -1;
}
I2CBus::~I2CBus() {}
int I2CBus::read_register(uint8_t device_address, uint register_address, uint8_t *buffer, uint8_t len) {
UNUSED(device_address);
UNUSED(register_address);
UNUSED(buffer);
UNUSED(len);
return -1;
}
int I2CBus::set_register(uint8_t device_address, uint register_address, uint8_t data) {
UNUSED(device_address);
UNUSED(register_address);
UNUSED(data);
return -1;
}
#endif

19
common/i2c.h Normal file
View File

@ -0,0 +1,19 @@
#pragma once
#include <cstdint>
#include <mutex>
#include <sys/types.h>
class I2CBus {
private:
int i2c_fd;
std::mutex m;
public:
I2CBus(uint8_t bus_id);
~I2CBus();
int read_register(uint8_t device_address, uint register_address, uint8_t *buffer, uint8_t len);
int set_register(uint8_t device_address, uint register_address, uint8_t data);
};

View File

@ -8,7 +8,6 @@ import uuid
import socket
import logging
import traceback
import numpy as np
from threading import local
from collections import OrderedDict
from contextlib import contextmanager
@ -16,8 +15,6 @@ from contextlib import contextmanager
LOG_TIMESTAMPS = "LOG_TIMESTAMPS" in os.environ
def json_handler(obj):
if isinstance(obj, np.bool_):
return bool(obj)
# if isinstance(obj, (datetime.date, datetime.time)):
# return obj.isoformat()
return repr(obj)
@ -199,6 +196,7 @@ class SwagLogger(logging.Logger):
co = f.f_code
filename = os.path.normcase(co.co_filename)
# TODO: is this pylint exception correct?
if filename == _srcfile:
f = f.f_back
continue

View File

@ -8,12 +8,12 @@ import functools
import threading
from cereal.messaging import PubMaster
from cereal.services import SERVICE_LIST
from openpilot.common.mock.generators import generate_livePose
from openpilot.common.mock.generators import generate_liveLocationKalman
from openpilot.common.realtime import Ratekeeper
MOCK_GENERATOR = {
"livePose": generate_livePose
"liveLocationKalman": generate_liveLocationKalman
}

View File

@ -1,14 +1,20 @@
from cereal import messaging
def generate_livePose():
msg = messaging.new_message('livePose')
meas = {'x': 0.0, 'y': 0.0, 'z': 0.0, 'xStd': 0.0, 'yStd': 0.0, 'zStd': 0.0, 'valid': True}
msg.livePose.orientationNED = meas
msg.livePose.velocityDevice = meas
msg.livePose.angularVelocityDevice = meas
msg.livePose.accelerationDevice = meas
msg.livePose.inputsOK = True
msg.livePose.posenetOK = True
msg.livePose.sensorsOK = True
LOCATION1 = (32.7174, -117.16277)
LOCATION2 = (32.7558, -117.2037)
LLK_DECIMATION = 10
RENDER_FRAMES = 15
DEFAULT_ITERATIONS = RENDER_FRAMES * LLK_DECIMATION
def generate_liveLocationKalman(location=LOCATION1):
msg = messaging.new_message('liveLocationKalman')
msg.liveLocationKalman.positionGeodetic = {'value': [*location, 0], 'std': [0., 0., 0.], 'valid': True}
msg.liveLocationKalman.positionECEF = {'value': [0., 0., 0.], 'std': [0., 0., 0.], 'valid': True}
msg.liveLocationKalman.calibratedOrientationNED = {'value': [0., 0., 0.], 'std': [0., 0., 0.], 'valid': True}
msg.liveLocationKalman.velocityCalibrated = {'value': [0., 0., 0.], 'std': [0., 0., 0.], 'valid': True}
msg.liveLocationKalman.status = 'valid'
msg.liveLocationKalman.gpsOK = True
return msg

19
common/numpy_fast.py Normal file
View File

@ -0,0 +1,19 @@
def clip(x, lo, hi):
return max(lo, min(hi, x))
def interp(x, xp, fp):
N = len(xp)
def get_interp(xv):
hi = 0
while hi < N and xv > xp[hi]:
hi += 1
low = hi - 1
return fp[-1] if hi == N and xv > xp[low] else (
fp[0] if hi == 0 else
(xv - xp[low]) * (fp[hi] - fp[low]) / (xp[hi] - xp[low]) + fp[low])
return [get_interp(v) for v in x] if hasattr(x, '__iter__') else get_interp(x)
def mean(x):
return sum(x) / len(x)

View File

@ -8,7 +8,6 @@
#include <csignal>
#include <unordered_map>
#include "common/params_keys.h"
#include "common/queue.h"
#include "common/swaglog.h"
#include "common/util.h"
@ -25,8 +24,8 @@ int fsync_dir(const std::string &path) {
int result = -1;
int fd = HANDLE_EINTR(open(path.c_str(), O_RDONLY, 0755));
if (fd >= 0) {
result = HANDLE_EINTR(fsync(fd));
HANDLE_EINTR(close(fd));
result = fsync(fd);
close(fd);
}
return result;
}
@ -77,6 +76,10 @@ std::string ensure_params_path(const std::string &prefix, const std::string &pat
class FileLock {
public:
FileLock(const std::string &fn) {
if (fn.rfind("/cache", 0) == 0 || fn.rfind("/data/params_backup", 0) == 0) {
return;
}
fd_ = HANDLE_EINTR(open(fn.c_str(), O_CREAT, 0775));
if (fd_ < 0 || HANDLE_EINTR(flock(fd_, LOCK_EX)) < 0) {
LOGE("Failed to lock file %s, errno=%d", fn.c_str(), errno);
@ -88,6 +91,518 @@ private:
int fd_ = -1;
};
std::unordered_map<std::string, uint32_t> keys = {
{"AccessToken", CLEAR_ON_MANAGER_START | DONT_LOG},
{"AlwaysOnDM", PERSISTENT},
{"ApiCache_Device", PERSISTENT},
{"ApiCache_NavDestinations", PERSISTENT},
{"AssistNowToken", PERSISTENT},
{"AthenadPid", PERSISTENT},
{"AthenadUploadQueue", PERSISTENT},
{"AthenadRecentlyViewedRoutes", PERSISTENT},
{"BootCount", PERSISTENT},
{"CalibrationParams", PERSISTENT},
{"CameraDebugExpGain", CLEAR_ON_MANAGER_START},
{"CameraDebugExpTime", CLEAR_ON_MANAGER_START},
{"CarBatteryCapacity", PERSISTENT},
{"CarParams", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"CarParamsCache", CLEAR_ON_MANAGER_START},
{"CarParamsPersistent", PERSISTENT},
{"CarParamsPrevRoute", PERSISTENT},
{"CarVin", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"CompletedTrainingVersion", PERSISTENT},
{"ControlsReady", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"CurrentBootlog", PERSISTENT},
{"CurrentRoute", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"DisableLogging", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"DisablePowerDown", PERSISTENT},
{"DisableUpdates", PERSISTENT},
{"DisengageOnAccelerator", PERSISTENT},
{"DmModelInitialized", CLEAR_ON_ONROAD_TRANSITION},
{"DongleId", PERSISTENT},
{"DoReboot", CLEAR_ON_MANAGER_START},
{"DoShutdown", CLEAR_ON_MANAGER_START},
{"DoUninstall", CLEAR_ON_MANAGER_START},
{"ExperimentalLongitudinalEnabled", PERSISTENT | DEVELOPMENT_ONLY},
{"ExperimentalMode", PERSISTENT},
{"ExperimentalModeConfirmed", PERSISTENT},
{"FirmwareQueryDone", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ForcePowerDown", PERSISTENT},
{"GitBranch", PERSISTENT},
{"GitCommit", PERSISTENT},
{"GitCommitDate", PERSISTENT},
{"GitDiff", PERSISTENT},
{"GithubSshKeys", PERSISTENT},
{"GithubUsername", PERSISTENT},
{"GitRemote", PERSISTENT},
{"GsmApn", PERSISTENT},
{"GsmMetered", PERSISTENT},
{"GsmRoaming", PERSISTENT},
{"HardwareSerial", PERSISTENT},
{"HasAcceptedTerms", PERSISTENT},
{"IMEI", PERSISTENT},
{"InstallDate", PERSISTENT},
{"IsDriverViewEnabled", CLEAR_ON_ONROAD_TRANSITION},
{"IsEngaged", PERSISTENT},
{"IsLdwEnabled", PERSISTENT},
{"IsMetric", PERSISTENT},
{"IsOffroad", CLEAR_ON_MANAGER_START},
{"IsOnroad", PERSISTENT},
{"IsRhdDetected", PERSISTENT},
{"IsReleaseBranch", CLEAR_ON_MANAGER_START},
{"IsTakingSnapshot", CLEAR_ON_MANAGER_START},
{"IsTestedBranch", CLEAR_ON_MANAGER_START},
{"JoystickDebugMode", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"LanguageSetting", PERSISTENT},
{"LastAthenaPingTime", CLEAR_ON_MANAGER_START},
{"LastGPSPosition", PERSISTENT},
{"LastManagerExitReason", CLEAR_ON_MANAGER_START},
{"LastOffroadStatusPacket", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"LastPowerDropDetected", CLEAR_ON_MANAGER_START},
{"LastUpdateException", CLEAR_ON_MANAGER_START},
{"LastUpdateTime", PERSISTENT},
{"LiveDelay", PERSISTENT},
{"LiveParameters", PERSISTENT},
{"LiveTorqueParameters", PERSISTENT | DONT_LOG},
{"LongitudinalPersonality", PERSISTENT},
{"NavDestination", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"NavDestinationWaypoints", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"NavPastDestinations", PERSISTENT},
{"NavSettingLeftSide", PERSISTENT},
{"NavSettingTime24h", PERSISTENT},
{"NetworkMetered", PERSISTENT},
{"ObdMultiplexingChanged", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ObdMultiplexingEnabled", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"Offroad_BadNvme", CLEAR_ON_MANAGER_START},
{"Offroad_CarUnrecognized", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"Offroad_ConnectivityNeeded", CLEAR_ON_MANAGER_START},
{"Offroad_ConnectivityNeededPrompt", CLEAR_ON_MANAGER_START},
{"Offroad_IsTakingSnapshot", CLEAR_ON_MANAGER_START},
{"Offroad_NeosUpdate", CLEAR_ON_MANAGER_START},
{"Offroad_NoFirmware", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"Offroad_Recalibration", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"Offroad_StorageMissing", CLEAR_ON_MANAGER_START},
{"Offroad_TemperatureTooHigh", CLEAR_ON_MANAGER_START},
{"Offroad_UnofficialHardware", CLEAR_ON_MANAGER_START},
{"Offroad_UpdateFailed", CLEAR_ON_MANAGER_START},
{"OpenpilotEnabledToggle", PERSISTENT},
{"PandaHeartbeatLost", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"PandaSomResetTriggered", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"PandaSignatures", CLEAR_ON_MANAGER_START},
{"PrimeType", PERSISTENT},
{"RecordFront", PERSISTENT},
{"RecordFrontLock", PERSISTENT}, // for the internal fleet
{"ReplayControlsState", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"RouteCount", PERSISTENT},
{"SecOCKey", PERSISTENT | DONT_LOG},
{"SnoozeUpdate", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"SshEnabled", PERSISTENT},
{"TermsVersion", PERSISTENT},
{"Timezone", PERSISTENT},
{"TrainingVersion", PERSISTENT},
{"UbloxAvailable", PERSISTENT},
{"UpdateAvailable", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"UpdateFailedCount", CLEAR_ON_MANAGER_START},
{"UpdaterAvailableBranches", PERSISTENT},
{"UpdaterCurrentDescription", CLEAR_ON_MANAGER_START},
{"UpdaterCurrentReleaseNotes", CLEAR_ON_MANAGER_START},
{"UpdaterFetchAvailable", CLEAR_ON_MANAGER_START},
{"UpdaterNewDescription", CLEAR_ON_MANAGER_START},
{"UpdaterNewReleaseNotes", CLEAR_ON_MANAGER_START},
{"UpdaterState", CLEAR_ON_MANAGER_START},
{"UpdaterTargetBranch", CLEAR_ON_MANAGER_START},
{"UpdaterLastFetchTime", PERSISTENT},
{"Version", PERSISTENT},
// FrogPilot parameters
{"AccelerationPath", PERSISTENT},
{"AccelerationProfile", PERSISTENT},
{"AdjacentLeadsUI", PERSISTENT},
{"AdjacentPath", PERSISTENT},
{"AdjacentPathMetrics", PERSISTENT},
{"AdvancedCustomUI", PERSISTENT},
{"AdvancedLateralTune", PERSISTENT},
{"AdvancedLongitudinalTune", PERSISTENT},
{"AggressiveFollow", PERSISTENT},
{"AggressiveJerkAcceleration", PERSISTENT},
{"AggressiveJerkDanger", PERSISTENT},
{"AggressiveJerkDeceleration", PERSISTENT},
{"AggressiveJerkSpeed", PERSISTENT},
{"AggressiveJerkSpeedDecrease", PERSISTENT},
{"AggressivePersonalityProfile", PERSISTENT},
{"AlertVolumeControl", PERSISTENT},
{"AlwaysOnLateral", PERSISTENT},
{"AlwaysOnLateralLKAS", PERSISTENT},
{"AlwaysOnLateralMain", PERSISTENT},
{"AMapKey1", PERSISTENT | DONT_LOG},
{"AMapKey2", PERSISTENT | DONT_LOG},
{"ApiCache_DriveStats", PERSISTENT},
{"AutomaticallyDownloadModels", PERSISTENT},
{"AutomaticUpdates", PERSISTENT},
{"AvailableModelNames", PERSISTENT},
{"AvailableModels", PERSISTENT},
{"BigMap", PERSISTENT},
{"BlacklistedModels", PERSISTENT},
{"BlindSpotMetrics", PERSISTENT},
{"BlindSpotPath", PERSISTENT},
{"BorderMetrics", PERSISTENT},
{"CalibratedLateralAcceleration", PERSISTENT},
{"CalibrationProgress", PERSISTENT},
{"CameraView", PERSISTENT},
{"CancelModelDownload", CLEAR_ON_MANAGER_START},
{"CancelThemeDownload", CLEAR_ON_MANAGER_START},
{"CarMake", PERSISTENT},
{"CarModel", PERSISTENT},
{"CarModelName", PERSISTENT},
{"CECurves", PERSISTENT},
{"CECurvesLead", PERSISTENT},
{"CELead", PERSISTENT},
{"CEModelStopTime", PERSISTENT},
{"CENavigation", PERSISTENT},
{"CENavigationIntersections", PERSISTENT},
{"CENavigationLead", PERSISTENT},
{"CENavigationTurns", PERSISTENT},
{"CESignalSpeed", PERSISTENT},
{"CESignalLaneDetection", PERSISTENT},
{"CESlowerLead", PERSISTENT},
{"CESpeed", PERSISTENT},
{"CESpeedLead", PERSISTENT},
{"CEStatus", CLEAR_ON_OFFROAD_TRANSITION},
{"CEStopLights", PERSISTENT},
{"CEStoppedLead", PERSISTENT},
{"ClusterOffset", PERSISTENT},
{"ColorToDownload", CLEAR_ON_MANAGER_START},
{"Compass", PERSISTENT},
{"ConditionalExperimental", PERSISTENT},
{"CurvatureData", PERSISTENT | DONT_LOG},
{"CurveSpeedController", PERSISTENT},
{"CustomAlerts", PERSISTENT},
{"CustomColors", PERSISTENT},
{"CustomCruise", PERSISTENT},
{"CustomCruiseLong", PERSISTENT},
{"CustomDistanceIcons", PERSISTENT},
{"CustomIcons", PERSISTENT},
{"CustomPersonalities", PERSISTENT},
{"CustomSignals", PERSISTENT},
{"CustomSounds", PERSISTENT},
{"CustomUI", PERSISTENT},
{"DebugMode", CLEAR_ON_OFFROAD_TRANSITION},
{"DecelerationProfile", PERSISTENT},
{"DeveloperMetrics", PERSISTENT},
{"DeveloperSidebar", PERSISTENT},
{"DeveloperSidebarMetric1", PERSISTENT},
{"DeveloperSidebarMetric2", PERSISTENT},
{"DeveloperSidebarMetric3", PERSISTENT},
{"DeveloperSidebarMetric4", PERSISTENT},
{"DeveloperSidebarMetric5", PERSISTENT},
{"DeveloperSidebarMetric6", PERSISTENT},
{"DeveloperSidebarMetric7", PERSISTENT},
{"DeveloperWidgets", PERSISTENT},
{"DeveloperUI", PERSISTENT},
{"DeviceManagement", PERSISTENT},
{"DeviceShutdown", PERSISTENT},
{"DisableOnroadUploads", PERSISTENT},
{"DisableOpenpilotLongitudinal", PERSISTENT},
{"DiscordUsername", PERSISTENT},
{"DistanceButtonControl", PERSISTENT},
{"DistanceIconToDownload", CLEAR_ON_MANAGER_START},
{"DisengageVolume", PERSISTENT},
{"DoToggleReset", PERSISTENT},
{"DoToggleResetStock", PERSISTENT},
{"DownloadableColors", PERSISTENT},
{"DownloadableDistanceIcons", PERSISTENT},
{"DownloadableIcons", PERSISTENT},
{"DownloadableSignals", PERSISTENT},
{"DownloadableSounds", PERSISTENT},
{"DownloadableWheels", PERSISTENT},
{"DownloadAllModels", CLEAR_ON_MANAGER_START},
{"DriverCamera", PERSISTENT},
{"DynamicPathWidth", PERSISTENT},
{"DynamicPedalsOnUI", PERSISTENT},
{"EngageVolume", PERSISTENT},
{"ExperimentalGMTune", PERSISTENT},
{"Fahrenheit", PERSISTENT},
{"FavoriteDestinations", PERSISTENT | DONT_LOG},
{"FlashPanda", CLEAR_ON_MANAGER_START},
{"ForceAutoTune", PERSISTENT},
{"ForceAutoTuneOff", PERSISTENT},
{"ForceFingerprint", PERSISTENT},
{"ForceMPHDashboard", PERSISTENT},
{"ForceOffroad", CLEAR_ON_MANAGER_START},
{"ForceOnroad", CLEAR_ON_MANAGER_START},
{"ForceStops", PERSISTENT},
{"ForceTorqueController", PERSISTENT},
{"FPSCounter", PERSISTENT},
{"FrogPilotCarParams", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"FrogPilotCarParamsPersistent", PERSISTENT},
{"FrogPilotDongleId", PERSISTENT | DONT_LOG},
{"FrogPilotDrives", PERSISTENT},
{"FrogPilotKilometers", PERSISTENT},
{"FrogPilotMinutes", PERSISTENT},
{"FrogPilotStats", PERSISTENT | DONT_LOG},
{"FrogPilotToggles", CLEAR_ON_MANAGER_START},
{"FrogPilotTogglesUpdated", CLEAR_ON_MANAGER_START},
{"FrogPilotTuningLevels", CLEAR_ON_MANAGER_START},
{"FrogsGoMoosTweak", PERSISTENT},
{"FullMap", PERSISTENT},
{"GasRegenCmd", PERSISTENT},
{"GoatScream", PERSISTENT},
{"GreenLightAlert", PERSISTENT},
{"HideAlerts", PERSISTENT},
{"HideLeadMarker", PERSISTENT},
{"HideMap", PERSISTENT},
{"HideMapIcon", PERSISTENT},
{"HideMaxSpeed", PERSISTENT},
{"HideSpeed", PERSISTENT},
{"HideSpeedLimit", PERSISTENT},
{"HigherBitrate", PERSISTENT},
{"HolidayThemes", PERSISTENT},
{"HondaAltTune", PERSISTENT},
{"HondaLowSpeedPedal", PERSISTENT},
{"HondaMaxBrake", PERSISTENT},
{"HumanAcceleration", PERSISTENT},
{"HumanFollowing", PERSISTENT},
{"HumanLaneChanges", PERSISTENT},
{"IconToDownload", CLEAR_ON_MANAGER_START},
{"IncreasedStoppedDistance", PERSISTENT},
{"IncreaseFollowingLowVisibility", PERSISTENT},
{"IncreaseFollowingRain", PERSISTENT},
{"IncreaseFollowingRainStorm", PERSISTENT},
{"IncreaseFollowingSnow", PERSISTENT},
{"IncreasedStoppedDistanceLowVisibility", PERSISTENT},
{"IncreasedStoppedDistanceRain", PERSISTENT},
{"IncreasedStoppedDistanceRainStorm", PERSISTENT},
{"IncreasedStoppedDistanceSnow", PERSISTENT},
{"IncreaseThermalLimits", PERSISTENT},
{"IssueReported", CLEAR_ON_MANAGER_START},
{"KonikDongleId", PERSISTENT},
{"KonikMinutes", PERSISTENT},
{"LaneChanges", PERSISTENT},
{"LaneChangeTime", PERSISTENT},
{"LaneDetectionWidth", PERSISTENT},
{"LaneLinesWidth", PERSISTENT},
{"LastMapsUpdate", PERSISTENT},
{"LateralTune", PERSISTENT},
{"LeadDepartingAlert", PERSISTENT},
{"LeadDetectionThreshold", PERSISTENT},
{"LeadInfo", PERSISTENT},
{"LKASButtonControl", PERSISTENT},
{"LockDoors", PERSISTENT},
{"LockDoorsTimer", PERSISTENT},
{"LongDistanceButtonControl", PERSISTENT},
{"LongitudinalActuatorDelay", PERSISTENT},
{"LongitudinalActuatorDelayStock", PERSISTENT},
{"LongitudinalTune", PERSISTENT},
{"LongPitch", PERSISTENT},
{"LoudBlindspotAlert", PERSISTENT},
{"LowVoltageShutdown", PERSISTENT},
{"ManualUpdateInitiated", CLEAR_ON_MANAGER_START},
{"MapAcceleration", PERSISTENT},
{"MapboxPublicKey", PERSISTENT | DONT_LOG},
{"MapBoxRequests", PERSISTENT},
{"MapboxSecretKey", PERSISTENT | DONT_LOG},
{"MapDeceleration", PERSISTENT},
{"MapdLogLevel", CLEAR_ON_MANAGER_START},
{"MapGears", PERSISTENT},
{"MapsSelected", PERSISTENT},
{"MapSpeedLimit", CLEAR_ON_MANAGER_START},
{"MapStyle", PERSISTENT},
{"MaxDesiredAcceleration", PERSISTENT},
{"MinimumBackupSize", PERSISTENT},
{"MinimumLaneChangeSpeed", PERSISTENT},
{"Model", PERSISTENT},
{"ModelDownloadProgress", CLEAR_ON_MANAGER_START},
{"ModelDrivesAndScores", PERSISTENT},
{"ModelRandomizer", PERSISTENT},
{"ModelToDownload", CLEAR_ON_MANAGER_START},
{"ModelUI", PERSISTENT},
{"ModelVersions", PERSISTENT},
{"NavigationUI", PERSISTENT},
{"NextMapSpeedLimit", CLEAR_ON_MANAGER_START},
{"NewLongAPI", PERSISTENT},
{"NNFF", PERSISTENT},
{"NNFFLite", PERSISTENT},
{"NNFFModelName", CLEAR_ON_OFFROAD_TRANSITION},
{"NoLogging", PERSISTENT},
{"NoUploads", PERSISTENT},
{"NudgelessLaneChange", PERSISTENT},
{"NumericalTemp", PERSISTENT},
{"Offset1", PERSISTENT},
{"Offset2", PERSISTENT},
{"Offset3", PERSISTENT},
{"Offset4", PERSISTENT},
{"Offset5", PERSISTENT},
{"Offset6", PERSISTENT},
{"Offset7", PERSISTENT},
{"OneLaneChange", PERSISTENT},
{"OnroadDistanceButton", PERSISTENT},
{"OnroadDistanceButtonPressed", CLEAR_ON_MANAGER_START},
{"openpilotMinutes", PERSISTENT},
{"OSMDownloadBounds", PERSISTENT},
{"OSMDownloadLocations", PERSISTENT},
{"OSMDownloadProgress", CLEAR_ON_MANAGER_START},
{"OverpassRequests", PERSISTENT},
{"PathEdgeWidth", PERSISTENT},
{"PathWidth", PERSISTENT},
{"PauseAOLOnBrake", PERSISTENT},
{"PauseLateralOnSignal", PERSISTENT},
{"PauseLateralSpeed", PERSISTENT},
{"PedalsOnUI", PERSISTENT},
{"PersonalizeOpenpilot", PERSISTENT},
{"PreferredSchedule", PERSISTENT},
{"PreviousSpeedLimit", PERSISTENT},
{"PromptDistractedVolume", PERSISTENT},
{"PromptVolume", PERSISTENT},
{"QOLLateral", PERSISTENT},
{"QOLLongitudinal", PERSISTENT},
{"QOLVisuals", PERSISTENT},
{"RadarTracksUI", PERSISTENT},
{"RainbowPath", PERSISTENT},
{"RandomEvents", PERSISTENT},
{"RandomThemes", PERSISTENT},
{"ReduceAccelerationLowVisibility", PERSISTENT},
{"ReduceAccelerationRain", PERSISTENT},
{"ReduceAccelerationRainStorm", PERSISTENT},
{"ReduceAccelerationSnow", PERSISTENT},
{"ReduceLateralAccelerationLowVisibility", PERSISTENT},
{"ReduceLateralAccelerationRain", PERSISTENT},
{"ReduceLateralAccelerationRainStorm", PERSISTENT},
{"ReduceLateralAccelerationSnow", PERSISTENT},
{"RefuseVolume", PERSISTENT},
{"RelaxedFollow", PERSISTENT},
{"RelaxedJerkAcceleration", PERSISTENT},
{"RelaxedJerkDanger", PERSISTENT},
{"RelaxedJerkDeceleration", PERSISTENT},
{"RelaxedJerkSpeed", PERSISTENT},
{"RelaxedJerkSpeedDecrease", PERSISTENT},
{"RelaxedPersonalityProfile", PERSISTENT},
{"ReverseCruise", PERSISTENT},
{"RoadEdgesWidth", PERSISTENT},
{"RoadName", CLEAR_ON_MANAGER_START},
{"RoadNameUI", PERSISTENT},
{"RotatingWheel", PERSISTENT},
{"ScreenBrightness", PERSISTENT},
{"ScreenBrightnessOnroad", PERSISTENT},
{"ScreenManagement", PERSISTENT},
{"ScreenRecorder", PERSISTENT},
{"ScreenTimeout", PERSISTENT},
{"ScreenTimeoutOnroad", PERSISTENT},
{"SearchInput", PERSISTENT},
{"SecOCKeys", PERSISTENT | DONT_LOG},
{"SetSpeedLimit", PERSISTENT},
{"SetSpeedOffset", PERSISTENT},
{"ShowCEMStatus", PERSISTENT},
{"ShowCPU", PERSISTENT},
{"ShowCSCStatus", PERSISTENT},
{"ShowGPU", PERSISTENT},
{"ShowIP", PERSISTENT},
{"ShowMemoryUsage", PERSISTENT},
{"ShownToggleDescriptions", PERSISTENT},
{"ShowSLCOffset", PERSISTENT},
{"ShowSpeedLimits", PERSISTENT},
{"ShowSteering", PERSISTENT},
{"ShowStoppingPoint", PERSISTENT},
{"ShowStoppingPointMetrics", PERSISTENT},
{"ShowStorageLeft", PERSISTENT},
{"ShowStorageUsed", PERSISTENT},
{"Sidebar", PERSISTENT},
{"SignalMetrics", PERSISTENT},
{"SignalToDownload", CLEAR_ON_MANAGER_START},
{"SLCConfirmation", PERSISTENT},
{"SLCConfirmationHigher", PERSISTENT},
{"SLCConfirmationLower", PERSISTENT},
{"SLCFallback", PERSISTENT},
{"SLCLookaheadHigher", PERSISTENT},
{"SLCLookaheadLower", PERSISTENT},
{"SLCMapboxFiller", PERSISTENT},
{"SLCOverride", PERSISTENT},
{"SLCPriority1", PERSISTENT},
{"SLCPriority2", PERSISTENT},
{"SLCPriority3", PERSISTENT},
{"SNGHack", PERSISTENT},
{"SoundToDownload", CLEAR_ON_MANAGER_START},
{"SpeedLimitAccepted", CLEAR_ON_MANAGER_START},
{"SpeedLimitChangedAlert", PERSISTENT},
{"SpeedLimitController", PERSISTENT},
{"SpeedLimitFiller", PERSISTENT},
{"SpeedLimits", PERSISTENT | DONT_LOG},
{"SpeedLimitsFiltered", PERSISTENT | DONT_LOG},
{"SpeedLimitSources", PERSISTENT},
{"StandardFollow", PERSISTENT},
{"StandardJerkAcceleration", PERSISTENT},
{"StandardJerkDanger", PERSISTENT},
{"StandardJerkDeceleration", PERSISTENT},
{"StandardJerkSpeed", PERSISTENT},
{"StandardJerkSpeedDecrease", PERSISTENT},
{"StandardPersonalityProfile", PERSISTENT},
{"StandbyMode", PERSISTENT},
{"StartAccel", PERSISTENT},
{"StartAccelStock", PERSISTENT},
{"StartupMessageBottom", PERSISTENT},
{"StartupMessageTop", PERSISTENT},
{"StaticPedalsOnUI", PERSISTENT},
{"SteerDelay", PERSISTENT},
{"SteerDelayStock", PERSISTENT},
{"SteerFriction", PERSISTENT},
{"SteerFrictionStock", PERSISTENT},
{"SteerLatAccel", PERSISTENT},
{"SteerLatAccelStock", PERSISTENT},
{"SteerKP", PERSISTENT},
{"SteerKPStock", PERSISTENT},
{"SteerRatio", PERSISTENT},
{"SteerRatioStock", PERSISTENT},
{"StockDongleId", PERSISTENT},
{"StopAccel", PERSISTENT},
{"StopAccelStock", PERSISTENT},
{"StoppingDecelRate", PERSISTENT},
{"StoppingDecelRateStock", PERSISTENT},
{"StoppedTimer", PERSISTENT},
{"SubaruSNG", PERSISTENT},
{"TacoTune", PERSISTENT},
{"TacoTuneHacks", PERSISTENT},
{"TestAlert", CLEAR_ON_MANAGER_START},
{"TetheringEnabled", PERSISTENT},
{"ThemeDownloadProgress", CLEAR_ON_MANAGER_START},
{"ThemesDownloaded", PERSISTENT},
{"TinygradUpdateAvailable", PERSISTENT},
{"ToyotaDoors", PERSISTENT},
{"TrafficFollow", PERSISTENT},
{"TrafficJerkAcceleration", PERSISTENT},
{"TrafficJerkDanger", PERSISTENT},
{"TrafficJerkDeceleration", PERSISTENT},
{"TrafficJerkSpeed", PERSISTENT},
{"TrafficJerkSpeedDecrease", PERSISTENT},
{"TrafficPersonalityProfile", PERSISTENT},
{"TuningLevel", PERSISTENT},
{"TuningLevelConfirmed", PERSISTENT},
{"TurnDesires", PERSISTENT},
{"UnlimitedLength", PERSISTENT},
{"UnlockDoors", PERSISTENT},
{"Updated", PERSISTENT},
{"UpdatedToggles", PERSISTENT},
{"UpdateSpeedLimits", CLEAR_ON_MANAGER_START},
{"UpdateSpeedLimitsStatus", CLEAR_ON_MANAGER_START},
{"UpdateTinygrad", CLEAR_ON_MANAGER_START},
{"UpdateWheelImage", CLEAR_ON_MANAGER_START},
{"UseActiveTheme", CLEAR_ON_MANAGER_START},
{"UseKonikServer", PERSISTENT},
{"UseSI", PERSISTENT},
{"UseVienna", PERSISTENT},
{"VEgoStarting", PERSISTENT},
{"VEgoStartingStock", PERSISTENT},
{"VEgoStopping", PERSISTENT},
{"VEgoStoppingStock", PERSISTENT},
{"VeryLongDistanceButtonControl", PERSISTENT},
{"VoltSNG", PERSISTENT},
{"WarningImmediateVolume", PERSISTENT},
{"WarningSoftVolume", PERSISTENT},
{"WeatherPresets", PERSISTENT},
{"WeatherToken", PERSISTENT | DONT_LOG},
{"WheelIcon", PERSISTENT},
{"WheelSpeed", PERSISTENT},
{"WheelToDownload", CLEAR_ON_MANAGER_START},
};
} // namespace
@ -115,16 +630,8 @@ bool Params::checkKey(const std::string &key) {
return keys.find(key) != keys.end();
}
ParamKeyFlag Params::getKeyFlag(const std::string &key) {
return static_cast<ParamKeyFlag>(keys[key].flags);
}
ParamKeyType Params::getKeyType(const std::string &key) {
return keys[key].type;
}
std::optional<std::string> Params::getKeyDefaultValue(const std::string &key) {
return keys[key].default_value;
return static_cast<ParamKeyType>(keys[key]);
}
int Params::put(const char* key, const char* value, size_t value_size) {
@ -148,7 +655,7 @@ int Params::put(const char* key, const char* value, size_t value_size) {
}
// fsync to force persist the changes.
if ((result = HANDLE_EINTR(fsync(tmp_fd))) < 0) break;
if ((result = fsync(tmp_fd)) < 0) break;
FileLock file_lock(params_path + "/.lock");
@ -203,17 +710,17 @@ std::map<std::string, std::string> Params::readAll() {
return util::read_files_in_dir(getParamPath());
}
void Params::clearAll(ParamKeyFlag key_flag) {
void Params::clearAll(ParamKeyType key_type) {
FileLock file_lock(params_path + "/.lock");
// 1) delete params of key_flag
// 1) delete params of key_type
// 2) delete files that are not defined in the keys.
if (DIR *d = opendir(getParamPath().c_str())) {
struct dirent *de = NULL;
while ((de = readdir(d))) {
if (de->d_type != DT_DIR) {
auto it = keys.find(de->d_name);
if (it == keys.end() || (it->second.flags & key_flag)) {
if (it == keys.end() || (it->second & key_type)) {
unlink(getParamPath(de->d_name).c_str());
}
}

View File

@ -2,7 +2,6 @@
#include <future>
#include <map>
#include <optional>
#include <string>
#include <tuple>
#include <utility>
@ -10,33 +9,16 @@
#include "common/queue.h"
enum ParamKeyFlag {
enum ParamKeyType {
PERSISTENT = 0x02,
CLEAR_ON_MANAGER_START = 0x04,
CLEAR_ON_ONROAD_TRANSITION = 0x08,
CLEAR_ON_OFFROAD_TRANSITION = 0x10,
DONT_LOG = 0x20,
DEVELOPMENT_ONLY = 0x40,
CLEAR_ON_IGNITION_ON = 0x80,
ALL = 0xFFFFFFFF
};
enum ParamKeyType {
STRING = 0, // must be utf-8 decodable
BOOL = 1,
INT = 2,
FLOAT = 3,
TIME = 4, // ISO 8601
JSON = 5,
BYTES = 6
};
struct ParamKeyAttributes {
uint32_t flags;
ParamKeyType type;
std::optional<std::string> default_value = std::nullopt;
};
class Params {
public:
explicit Params(const std::string &path = {});
@ -47,22 +29,28 @@ public:
std::vector<std::string> allKeys() const;
bool checkKey(const std::string &key);
ParamKeyFlag getKeyFlag(const std::string &key);
ParamKeyType getKeyType(const std::string &key);
std::optional<std::string> getKeyDefaultValue(const std::string &key);
inline std::string getParamPath(const std::string &key = {}) {
return params_path + params_prefix + (key.empty() ? "" : "/" + key);
}
// Delete a value
int remove(const std::string &key);
void clearAll(ParamKeyFlag flag);
void clearAll(ParamKeyType type);
// helpers for reading values
std::string get(const std::string &key, bool block = false);
inline bool getBool(const std::string &key, bool block = false) {
return get(key, block) == "1";
}
inline int getInt(const std::string &key, bool block = false) {
std::string value = get(key, block);
return value.empty() ? 0 : std::stoi(value);
}
inline float getFloat(const std::string &key, bool block = false) {
std::string value = get(key, block);
return value.empty() ? 0.0 : std::stof(value);
}
std::map<std::string, std::string> readAll();
// helpers for writing values
@ -73,10 +61,22 @@ public:
inline int putBool(const std::string &key, bool val) {
return put(key.c_str(), val ? "1" : "0", 1);
}
inline int putInt(const std::string &key, int val) {
return put(key.c_str(), std::to_string(val).c_str(), std::to_string(val).size());
}
inline int putFloat(const std::string &key, float val) {
return put(key.c_str(), std::to_string(val).c_str(), std::to_string(val).size());
}
void putNonBlocking(const std::string &key, const std::string &val);
inline void putBoolNonBlocking(const std::string &key, bool val) {
putNonBlocking(key, val ? "1" : "0");
}
inline void putIntNonBlocking(const std::string &key, int val) {
putNonBlocking(key, std::to_string(val));
}
inline void putFloatNonBlocking(const std::string &key, float val) {
putNonBlocking(key, std::to_string(val));
}
private:
void asyncWriteThread();

View File

@ -1,6 +1,5 @@
from openpilot.common.params_pyx import Params, ParamKeyFlag, ParamKeyType, UnknownKeyName
from openpilot.common.params_pyx import Params, ParamKeyType, UnknownKeyName
assert Params
assert ParamKeyFlag
assert ParamKeyType
assert UnknownKeyName

View File

@ -1,166 +0,0 @@
#pragma once
#include <string>
#include <unordered_map>
#include "cereal/gen/cpp/log.capnp.h"
inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
{"AccessToken", {CLEAR_ON_MANAGER_START | DONT_LOG, STRING}},
{"AdbEnabled", {PERSISTENT, BOOL}},
{"AlwaysOnDM", {PERSISTENT, BOOL}},
{"ApiCache_Device", {PERSISTENT, STRING}},
{"ApiCache_FirehoseStats", {PERSISTENT, JSON}},
{"AssistNowToken", {PERSISTENT, STRING}},
{"AthenadPid", {PERSISTENT, INT}},
{"AthenadUploadQueue", {PERSISTENT, JSON}},
{"AthenadRecentlyViewedRoutes", {PERSISTENT, STRING}},
{"BootCount", {PERSISTENT, INT}},
{"CalibrationParams", {PERSISTENT, BYTES}},
{"CameraDebugExpGain", {CLEAR_ON_MANAGER_START, STRING}},
{"CameraDebugExpTime", {CLEAR_ON_MANAGER_START, STRING}},
{"CarBatteryCapacity", {PERSISTENT, INT}},
{"CarParams", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BYTES}},
{"CarParamsCache", {CLEAR_ON_MANAGER_START, BYTES}},
{"CarParamsPersistent", {PERSISTENT, BYTES}},
{"CarParamsPrevRoute", {PERSISTENT, BYTES}},
{"CompletedTrainingVersion", {PERSISTENT, STRING, "0"}},
{"ControlsReady", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"CurrentBootlog", {PERSISTENT, STRING}},
{"CurrentRoute", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, STRING}},
{"DisableLogging", {PERSISTENT, BOOL, "0"}},
{"DisablePowerDown", {PERSISTENT, BOOL}},
{"DisableUpdates", {PERSISTENT, BOOL}},
{"DisengageOnAccelerator", {PERSISTENT, BOOL, "0"}},
{"DongleId", {PERSISTENT, STRING}},
{"DoReboot", {CLEAR_ON_MANAGER_START, BOOL}},
{"DoShutdown", {CLEAR_ON_MANAGER_START, BOOL}},
{"DoUninstall", {CLEAR_ON_MANAGER_START, BOOL}},
{"DriverTooDistracted", {CLEAR_ON_MANAGER_START | CLEAR_ON_IGNITION_ON, BOOL}},
{"AlphaLongitudinalEnabled", {PERSISTENT | DEVELOPMENT_ONLY, BOOL}},
{"ExperimentalMode", {PERSISTENT, BOOL}},
{"ExperimentalModeConfirmed", {PERSISTENT, BOOL}},
{"FirmwareQueryDone", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"ForcePowerDown", {PERSISTENT, BOOL}},
{"GitBranch", {PERSISTENT, STRING}},
{"GitCommit", {PERSISTENT, STRING}},
{"GitCommitDate", {PERSISTENT, STRING}},
{"GitDiff", {PERSISTENT, STRING}},
{"GithubSshKeys", {PERSISTENT, STRING}},
{"GithubUsername", {PERSISTENT, STRING}},
{"GitRemote", {PERSISTENT, STRING}},
{"GsmApn", {PERSISTENT, STRING}},
{"GsmMetered", {PERSISTENT, BOOL, "1"}},
{"GsmRoaming", {PERSISTENT, BOOL}},
{"HardwareSerial", {PERSISTENT, STRING}},
{"HasAcceptedTerms", {PERSISTENT, STRING, "0"}},
{"InstallDate", {PERSISTENT, TIME}},
{"IsDriverViewEnabled", {CLEAR_ON_MANAGER_START, BOOL}},
{"IsEngaged", {PERSISTENT, BOOL}},
{"IsLdwEnabled", {PERSISTENT, BOOL}},
{"IsMetric", {PERSISTENT, BOOL}},
{"IsOffroad", {CLEAR_ON_MANAGER_START, BOOL}},
{"IsOnroad", {PERSISTENT, BOOL}},
{"IsRhdDetected", {PERSISTENT, BOOL}},
{"IsReleaseBranch", {CLEAR_ON_MANAGER_START, BOOL}},
{"IsTakingSnapshot", {CLEAR_ON_MANAGER_START, BOOL}},
{"IsTestedBranch", {CLEAR_ON_MANAGER_START, BOOL}},
{"JoystickDebugMode", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, BOOL}},
{"LanguageSetting", {PERSISTENT, STRING, "zh-CHS"}},
{"LastAthenaPingTime", {CLEAR_ON_MANAGER_START, INT}},
{"LastGPSPosition", {PERSISTENT, STRING}},
{"LastManagerExitReason", {CLEAR_ON_MANAGER_START, STRING}},
{"LastOffroadStatusPacket", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, JSON}},
{"LastPowerDropDetected", {CLEAR_ON_MANAGER_START, STRING}},
{"LastUpdateException", {CLEAR_ON_MANAGER_START, STRING}},
{"LastUpdateRouteCount", {PERSISTENT, INT, "0"}},
{"LastUpdateTime", {PERSISTENT, TIME}},
{"LastUpdateUptimeOnroad", {PERSISTENT, FLOAT, "0.0"}},
{"LiveDelay", {PERSISTENT, BYTES}},
{"LiveParameters", {PERSISTENT, JSON}},
{"LiveParametersV2", {PERSISTENT, BYTES}},
{"LiveTorqueParameters", {PERSISTENT | DONT_LOG, BYTES}},
{"LocationFilterInitialState", {PERSISTENT, BYTES}},
{"LongitudinalManeuverMode", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, BOOL}},
{"LongitudinalPersonality", {PERSISTENT, INT, std::to_string(static_cast<int>(cereal::LongitudinalPersonality::STANDARD))}},
{"NetworkMetered", {PERSISTENT, BOOL}},
{"ObdMultiplexingChanged", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"ObdMultiplexingEnabled", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"Offroad_CarUnrecognized", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, JSON}},
{"Offroad_ConnectivityNeeded", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_ConnectivityNeededPrompt", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_ExcessiveActuation", {PERSISTENT, JSON}},
{"Offroad_IsTakingSnapshot", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_NeosUpdate", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_NoFirmware", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, JSON}},
{"Offroad_Recalibration", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, JSON}},
{"Offroad_TemperatureTooHigh", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_UnregisteredHardware", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_UpdateFailed", {CLEAR_ON_MANAGER_START, JSON}},
{"OnroadCycleRequested", {CLEAR_ON_MANAGER_START, BOOL}},
{"OpenpilotEnabledToggle", {PERSISTENT, BOOL, "1"}},
{"PandaHeartbeatLost", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, BOOL}},
{"PandaSomResetTriggered", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, BOOL}},
{"PandaSignatures", {CLEAR_ON_MANAGER_START, BYTES}},
{"PrimeType", {PERSISTENT, INT}},
{"RecordAudio", {PERSISTENT, BOOL}},
{"RecordAudioFeedback", {PERSISTENT, BOOL, "0"}},
{"RecordFront", {PERSISTENT, BOOL}},
{"RecordFrontLock", {PERSISTENT, BOOL}}, // for the internal fleet
{"SecOCKey", {PERSISTENT | DONT_LOG, STRING}},
{"RouteCount", {PERSISTENT, INT, "0"}},
{"SnoozeUpdate", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, BOOL}},
{"SshEnabled", {PERSISTENT, BOOL}},
{"TermsVersion", {PERSISTENT, STRING}},
{"TrainingVersion", {PERSISTENT, STRING}},
{"UbloxAvailable", {PERSISTENT, BOOL}},
{"UpdateAvailable", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"UpdateFailedCount", {CLEAR_ON_MANAGER_START, INT}},
{"UpdaterAvailableBranches", {PERSISTENT, STRING}},
{"UpdaterCurrentDescription", {CLEAR_ON_MANAGER_START, STRING}},
{"UpdaterCurrentReleaseNotes", {CLEAR_ON_MANAGER_START, BYTES}},
{"UpdaterFetchAvailable", {CLEAR_ON_MANAGER_START, BOOL}},
{"UpdaterNewDescription", {CLEAR_ON_MANAGER_START, STRING}},
{"UpdaterNewReleaseNotes", {CLEAR_ON_MANAGER_START, BYTES}},
{"UpdaterState", {CLEAR_ON_MANAGER_START, STRING}},
{"UpdaterTargetBranch", {CLEAR_ON_MANAGER_START, STRING}},
{"UpdaterLastFetchTime", {PERSISTENT, TIME}},
{"UptimeOffroad", {PERSISTENT, FLOAT, "0.0"}},
{"UptimeOnroad", {PERSISTENT, FLOAT, "0.0"}},
{"Version", {PERSISTENT, STRING}},
{"dp_dev_last_log", {CLEAR_ON_ONROAD_TRANSITION, STRING}},
{"dp_dev_reset_conf", {CLEAR_ON_MANAGER_START, BOOL, "0"}},
{"dp_dev_is_rhd", {PERSISTENT, BOOL, "0"}},
{"dp_dev_monitoring_disabled", {PERSISTENT, BOOL, "0"}},
{"dp_dev_beep", {PERSISTENT, BOOL, "0"}},
{"dp_lat_alka", {PERSISTENT, BOOL, "0"}},
{"dp_ui_display_mode", {PERSISTENT, INT, "0"}},
{"dp_dev_model_selected", {PERSISTENT, STRING}},
{"dp_dev_model_list", {PERSISTENT, STRING}},
{"dp_lat_lca_speed", {PERSISTENT, INT, "20"}},
{"dp_lat_lca_auto_sec", {PERSISTENT, FLOAT, "0.0"}},
{"dp_dev_go_off_road", {CLEAR_ON_MANAGER_START, BOOL}},
{"dp_ui_hide_hud_speed_kph", {PERSISTENT, INT, "0"}},
{"dp_lon_ext_radar", {PERSISTENT, BOOL, "0"}},
{"dp_lat_road_edge_detection", {PERSISTENT, BOOL, "0"}},
{"dp_ui_rainbow", {PERSISTENT, BOOL, "0"}},
{"dp_lon_acm", {PERSISTENT, BOOL, "0"}},
{"dp_lon_aem", {PERSISTENT, BOOL, "0"}},
{"dp_lon_dtsc", {PERSISTENT, BOOL, "0"}},
{"dp_dev_audible_alert_mode", {PERSISTENT, INT, "0"}},
{"dp_dev_auto_shutdown_in", {PERSISTENT, INT, "-5"}},
{"dp_ui_lead", {PERSISTENT, INT, "0"}},
{"dp_dev_dashy", {PERSISTENT, INT, "0"}},
{"dp_dev_delay_loggerd", {PERSISTENT, INT, "0"}},
{"dp_dev_disable_connect", {PERSISTENT, BOOL, "0"}},
{"dp_dev_tethering", {PERSISTENT, BOOL, "0"}},
{"dp_toyota_door_auto_lock_unlock", {PERSISTENT, BOOL, "0"}},
{"dp_toyota_tss1_sng", {PERSISTENT, BOOL, "0"}},
{"dp_toyota_stock_lon", {PERSISTENT, BOOL, "0"}},
{"dp_vag_a0_sng", {PERSISTENT, BOOL, "0"}},
{"dp_vag_pq_steering_patch", {PERSISTENT, BOOL, "0"}},
{"dp_vag_avoid_eps_lockout", {PERSISTENT, BOOL, "0"}},
// 驾驶员监控灵敏度
{"DistractionDetectionLevel", {PERSISTENT, INT, "1"}},
};

19684
common/params_pyx.cpp Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,69 +1,38 @@
# distutils: language = c++
# cython: language_level = 3
import builtins
import datetime
import json
from libcpp cimport bool
from libcpp.string cimport string
from libcpp.vector cimport vector
from libcpp.optional cimport optional
from openpilot.common.swaglog import cloudlog
cdef extern from "common/params.h":
cpdef enum ParamKeyFlag:
cpdef enum ParamKeyType:
PERSISTENT
CLEAR_ON_MANAGER_START
CLEAR_ON_ONROAD_TRANSITION
CLEAR_ON_OFFROAD_TRANSITION
DEVELOPMENT_ONLY
CLEAR_ON_IGNITION_ON
ALL
cpdef enum ParamKeyType:
STRING
BOOL
INT
FLOAT
TIME
JSON
BYTES
cdef cppclass c_Params "Params":
c_Params(string) except + nogil
string get(string, bool) nogil
bool getBool(string, bool) nogil
int getInt(string, bool) nogil
float getFloat(string, bool) nogil
int remove(string) nogil
int put(string, string) nogil
void putNonBlocking(string, string) nogil
void putBoolNonBlocking(string, bool) nogil
void putIntNonBlocking(string, int) nogil
void putFloatNonBlocking(string, float) nogil
int putBool(string, bool) nogil
int putInt(string, int) nogil
int putFloat(string, float) nogil
bool checkKey(string) nogil
ParamKeyType getKeyType(string) nogil
optional[string] getKeyDefaultValue(string) nogil
string getParamPath(string) nogil
void clearAll(ParamKeyFlag)
void clearAll(ParamKeyType)
vector[string] allKeys()
PYTHON_2_CPP = {
(str, STRING): lambda v: v,
(builtins.bool, BOOL): lambda v: "1" if v else "0",
(int, INT): str,
(float, FLOAT): str,
(datetime.datetime, TIME): lambda v: v.isoformat(),
(dict, JSON): json.dumps,
(list, JSON): json.dumps,
(bytes, BYTES): lambda v: v,
}
CPP_2_PYTHON = {
STRING: lambda v: v.decode("utf-8"),
BOOL: lambda v: v == b"1",
INT: int,
FLOAT: float,
TIME: lambda v: datetime.datetime.fromisoformat(v.decode("utf-8")),
JSON: json.loads,
BYTES: lambda v: v,
}
def ensure_bytes(v):
return v.encode() if isinstance(v, str) else v
@ -73,22 +42,17 @@ class UnknownKeyName(Exception):
cdef class Params:
cdef c_Params* p
cdef str d
def __cinit__(self, d=""):
cdef string path = <string>d.encode()
with nogil:
self.p = new c_Params(path)
self.d = d
def __reduce__(self):
return (type(self), (self.d,))
def __dealloc__(self):
del self.p
def clear_all(self, tx_flag=ParamKeyFlag.ALL):
self.p.clearAll(tx_flag)
def clear_all(self, tx_type=ParamKeyType.ALL):
self.p.clearAll(tx_type)
def check_key(self, key):
key = ensure_bytes(key)
@ -96,38 +60,21 @@ cdef class Params:
raise UnknownKeyName(key)
return key
def python2cpp(self, proposed_type, expected_type, value, key):
cast = PYTHON_2_CPP.get((proposed_type, expected_type))
if cast:
return cast(value)
raise TypeError(f"Type mismatch while writing param {key}: {proposed_type=} {expected_type=} {value=}")
def _cpp2python(self, t, value, default, key):
if value is None:
return None
try:
return CPP_2_PYTHON[t](value)
except (KeyError, TypeError, ValueError):
cloudlog.warning(f"Failed to cast param {key} with {value=} from type {t=}")
return self._cpp2python(t, default, None, key)
def get(self, key, bool block=False, bool return_default=False):
def get(self, key, bool block=False, encoding=None):
cdef string k = self.check_key(key)
cdef ParamKeyType t = self.p.getKeyType(k)
cdef optional[string] default = self.p.getKeyDefaultValue(k)
cdef string val
with nogil:
val = self.p.get(k, block)
default_val = (default.value() if default.has_value() else None) if return_default else None
if val == b"":
if block:
# If we got no value while running in blocked mode
# it means we got an interrupt while waiting
raise KeyboardInterrupt
else:
return self._cpp2python(t, default_val, None, key)
return self._cpp2python(t, val, default_val, key)
return None
return val if encoding is None else val.decode(encoding)
def get_bool(self, key, bool block=False):
cdef string k = self.check_key(key)
@ -136,10 +83,19 @@ cdef class Params:
r = self.p.getBool(k, block)
return r
def _put_cast(self, key, dat):
def get_int(self, key, bool block=False):
cdef string k = self.check_key(key)
cdef ParamKeyType t = self.p.getKeyType(k)
return ensure_bytes(self.python2cpp(type(dat), t, dat, key))
cdef int r
with nogil:
r = self.p.getInt(k, block)
return r
def get_float(self, key, bool block=False):
cdef string k = self.check_key(key)
cdef float r
with nogil:
r = self.p.getFloat(k, block)
return r
def put(self, key, dat):
"""
@ -149,7 +105,7 @@ cdef class Params:
in general try to avoid writing params as much as possible.
"""
cdef string k = self.check_key(key)
cdef string dat_bytes = self._put_cast(key, dat)
cdef string dat_bytes = ensure_bytes(dat)
with nogil:
self.p.put(k, dat_bytes)
@ -158,9 +114,19 @@ cdef class Params:
with nogil:
self.p.putBool(k, val)
def put_int(self, key, int val):
cdef string k = self.check_key(key)
with nogil:
self.p.putInt(k, val)
def put_float(self, key, float val):
cdef string k = self.check_key(key)
with nogil:
self.p.putFloat(k, val)
def put_nonblocking(self, key, dat):
cdef string k = self.check_key(key)
cdef string dat_bytes = self._put_cast(key, dat)
cdef string dat_bytes = ensure_bytes(dat)
with nogil:
self.p.putNonBlocking(k, dat_bytes)
@ -169,6 +135,16 @@ cdef class Params:
with nogil:
self.p.putBoolNonBlocking(k, val)
def put_int_nonblocking(self, key, int val):
cdef string k = self.check_key(key)
with nogil:
self.p.putIntNonBlocking(k, val)
def put_float_nonblocking(self, key, float val):
cdef string k = self.check_key(key)
with nogil:
self.p.putFloatNonBlocking(k, val)
def remove(self, key):
cdef string k = self.check_key(key)
with nogil:
@ -178,19 +154,5 @@ cdef class Params:
cdef string key_bytes = ensure_bytes(key)
return self.p.getParamPath(key_bytes).decode("utf-8")
def get_type(self, key):
return self.p.getKeyType(self.check_key(key))
def all_keys(self):
return self.p.allKeys()
def get_default_value(self, key):
cdef string k = self.check_key(key)
cdef ParamKeyType t = self.p.getKeyType(k)
cdef optional[string] default = self.p.getKeyDefaultValue(k)
return self._cpp2python(t, default.value(), None, key) if default.has_value() else None
def cpp2python(self, key, value):
cdef string k = self.check_key(key)
cdef ParamKeyType t = self.p.getKeyType(k)
return self._cpp2python(t, value, None, key)

BIN
common/params_pyx.so Executable file

Binary file not shown.

View File

@ -1,64 +0,0 @@
import numpy as np
from numbers import Number
class PIDController:
def __init__(self, k_p, k_i, k_f=0., k_d=0., pos_limit=1e308, neg_limit=-1e308, rate=100):
self._k_p = k_p
self._k_i = k_i
self._k_d = k_d
self.k_f = k_f # feedforward gain
if isinstance(self._k_p, Number):
self._k_p = [[0], [self._k_p]]
if isinstance(self._k_i, Number):
self._k_i = [[0], [self._k_i]]
if isinstance(self._k_d, Number):
self._k_d = [[0], [self._k_d]]
self.set_limits(pos_limit, neg_limit)
self.i_dt = 1.0 / rate
self.speed = 0.0
self.reset()
@property
def k_p(self):
return np.interp(self.speed, self._k_p[0], self._k_p[1])
@property
def k_i(self):
return np.interp(self.speed, self._k_i[0], self._k_i[1])
@property
def k_d(self):
return np.interp(self.speed, self._k_d[0], self._k_d[1])
def reset(self):
self.p = 0.0
self.i = 0.0
self.d = 0.0
self.f = 0.0
self.control = 0
def set_limits(self, pos_limit, neg_limit):
self.pos_limit = pos_limit
self.neg_limit = neg_limit
def update(self, error, error_rate=0.0, speed=0.0, feedforward=0., freeze_integrator=False):
self.speed = speed
self.p = self.k_p * float(error)
self.d = self.k_d * error_rate
self.f = self.k_f * feedforward
if not freeze_integrator:
i = self.i + self.k_i * self.i_dt * error
# Don't allow windup if already clipping
test_control = self.p + i + self.d + self.f
i_upperbound = self.i if test_control > self.pos_limit else self.pos_limit
i_lowerbound = self.i if test_control < self.neg_limit else self.neg_limit
self.i = np.clip(i, i_lowerbound, i_upperbound)
control = self.p + self.i + self.d + self.f
self.control = np.clip(control, self.neg_limit, self.pos_limit)
return self.control

View File

@ -13,7 +13,7 @@ public:
if (prefix.empty()) {
prefix = util::random_string(15);
}
msgq_path = Path::shm_path() + "/" + prefix;
msgq_path = "/dev/shm/" + prefix;
bool ret = util::create_directories(msgq_path, 0777);
assert(ret);
setenv("OPENPILOT_PREFIX", prefix.c_str(), 1);

View File

@ -9,19 +9,20 @@ from openpilot.system.hardware.hw import Paths
from openpilot.system.hardware.hw import DEFAULT_DOWNLOAD_CACHE_ROOT
class OpenpilotPrefix:
def __init__(self, prefix: str = None, create_dirs_on_enter: bool = True, clean_dirs_on_exit: bool = True, shared_download_cache: bool = False):
def __init__(self, prefix: str = None, clean_dirs_on_exit: bool = True, shared_download_cache: bool = False):
self.prefix = prefix if prefix else str(uuid.uuid4().hex[0:15])
self.msgq_path = os.path.join(Paths.shm_path(), self.prefix)
self.create_dirs_on_enter = create_dirs_on_enter
self.msgq_path = os.path.join('/dev/shm', self.prefix)
self.clean_dirs_on_exit = clean_dirs_on_exit
self.shared_download_cache = shared_download_cache
def __enter__(self):
self.original_prefix = os.environ.get('OPENPILOT_PREFIX', None)
os.environ['OPENPILOT_PREFIX'] = self.prefix
if self.create_dirs_on_enter:
self.create_dirs()
try:
os.mkdir(self.msgq_path)
except FileExistsError:
pass
os.makedirs(Paths.log_root(), exist_ok=True)
if self.shared_download_cache:
os.environ["COMMA_CACHE"] = DEFAULT_DOWNLOAD_CACHE_ROOT
@ -39,13 +40,6 @@ class OpenpilotPrefix:
pass
return False
def create_dirs(self):
try:
os.mkdir(self.msgq_path)
except FileExistsError:
pass
os.makedirs(Paths.log_root(), exist_ok=True)
def clean_dirs(self):
symlink_path = Params().get_param_path()
if os.path.exists(symlink_path):

View File

@ -1,12 +1,11 @@
"""Utilities for reading real time clocks and keeping soft real time constraints."""
import gc
import os
import sys
import time
from collections import deque
from setproctitle import getproctitle
from openpilot.common.util import MovingAverage
from openpilot.system.hardware import PC
@ -28,15 +27,19 @@ class Priority:
CTRL_HIGH = 53
def set_realtime_priority(level: int) -> None:
if not PC:
os.sched_setscheduler(0, os.SCHED_FIFO, os.sched_param(level))
def set_core_affinity(cores: list[int]) -> None:
if sys.platform == 'linux' and not PC:
if not PC:
os.sched_setaffinity(0, cores)
def config_realtime_process(cores: int | list[int], priority: int) -> None:
gc.disable()
if sys.platform == 'linux' and not PC:
os.sched_setscheduler(0, os.SCHED_FIFO, os.sched_param(priority))
set_realtime_priority(priority)
c = cores if isinstance(cores, list) else [cores, ]
set_core_affinity(c)
@ -45,15 +48,13 @@ class Ratekeeper:
def __init__(self, rate: float, print_delay_threshold: float | None = 0.0) -> None:
"""Rate in Hz for ratekeeping. print_delay_threshold must be nonnegative."""
self._interval = 1. / rate
self._next_frame_time = time.monotonic() + self._interval
self._print_delay_threshold = print_delay_threshold
self._frame = 0
self._remaining = 0.0
self._process_name = getproctitle()
self._last_monitor_time = -1.
self._next_frame_time = -1.
self.avg_dt = MovingAverage(100)
self.avg_dt.add_value(self._interval)
self._dts = deque([self._interval], maxlen=100)
self._last_monitor_time = time.monotonic()
@property
def frame(self) -> int:
@ -65,8 +66,9 @@ class Ratekeeper:
@property
def lagging(self) -> bool:
avg_dt = sum(self._dts) / len(self._dts)
expected_dt = self._interval * (1 / 0.9)
return self.avg_dt.get_average() > expected_dt
return avg_dt > expected_dt
# Maintain loop rate by calling this at the end of each loop
def keep_time(self) -> bool:
@ -77,13 +79,9 @@ class Ratekeeper:
# Monitors the cumulative lag, but does not enforce a rate
def monitor_time(self) -> bool:
if self._last_monitor_time < 0:
self._next_frame_time = time.monotonic() + self._interval
self._last_monitor_time = time.monotonic()
prev = self._last_monitor_time
self._last_monitor_time = time.monotonic()
self.avg_dt.add_value(self._last_monitor_time - prev)
self._dts.append(self._last_monitor_time - prev)
lagged = False
remaining = self._next_frame_time - time.monotonic()

View File

@ -1,6 +1,4 @@
import subprocess
from contextlib import contextmanager
from subprocess import Popen, PIPE, TimeoutExpired
def run_cmd(cmd: list[str], cwd=None, env=None) -> str:
@ -13,16 +11,3 @@ def run_cmd_default(cmd: list[str], default: str = "", cwd=None, env=None) -> st
except subprocess.CalledProcessError:
return default
@contextmanager
def managed_proc(cmd: list[str], env: dict[str, str]):
proc = Popen(cmd, env=env, stdout=PIPE, stderr=PIPE)
try:
yield proc
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except TimeoutExpired:
proc.kill()

4
common/spinner.py Executable file → Normal file
View File

@ -6,9 +6,9 @@ from openpilot.common.basedir import BASEDIR
class Spinner:
def __init__(self):
try:
self.spinner_proc = subprocess.Popen(["./spinner.py"],
self.spinner_proc = subprocess.Popen(["./spinner"],
stdin=subprocess.PIPE,
cwd=os.path.join(BASEDIR, "system", "ui"),
cwd=os.path.join(BASEDIR, "selfdrive", "ui"),
close_fds=True)
except OSError:
self.spinner_proc = None

View File

@ -26,9 +26,6 @@ public:
zmq_setsockopt(sock, ZMQ_LINGER, &timeout, sizeof(timeout));
zmq_connect(sock, Path::swaglog_ipc().c_str());
// workaround for https://github.com/dropbox/json11/issues/38
setlocale(LC_NUMERIC, "C");
print_level = CLOUDLOG_WARNING;
if (const char* print_lvl = getenv("LOGPRINT")) {
if (strcmp(print_lvl, "debug") == 0) {

View File

@ -104,15 +104,6 @@ class UnixDomainSocketHandler(logging.Handler):
pass
class ForwardingHandler(logging.Handler):
def __init__(self, target_logger):
super().__init__()
self.target_logger = target_logger
def emit(self, record):
self.target_logger.handle(record)
def add_file_handler(log):
"""
Function to add the file log handler to swaglog.

View File

@ -1 +0,0 @@
test_common

View File

@ -1,19 +0,0 @@
import os
from uuid import uuid4
from openpilot.common.file_helpers import atomic_write_in_dir
class TestFileHelpers:
def run_atomic_write_func(self, atomic_write_func):
path = f"/tmp/tmp{uuid4()}"
with atomic_write_func(path) as f:
f.write("test")
assert not os.path.exists(path)
with open(path) as f:
assert f.read() == "test"
os.remove(path)
def test_atomic_write_in_dir(self):
self.run_atomic_write_func(atomic_write_in_dir)

Some files were not shown because too many files have changed in this diff Show More