Commit graph

884 commits

Author SHA1 Message Date
Thomas Petazzoni
3c41681075 static/messages.json: update
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2025-07-25 12:25:38 +02:00
Thomas Petazzoni
092d4496de static/messages.json: update
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2025-07-25 12:25:34 +02:00
Théo Lebrun
c0c5d2283c query: drop query.supports_dts_comp() getter method
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-03-07 16:16:34 +01:00
Théo Lebrun
e1fecd1c3f query: fix comment describing get_versions()
The comment was plain wrong. It said we returned a list.
We return nested OrderedDict structures.

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-03-07 16:16:34 +01:00
Franciszek Stachura
6f03eee42c CHANGELOG: Add info about changes in query tool 2025-03-07 16:16:34 +01:00
Franciszek Stachura
0dcf9d4197 query: Move CLI to utils/query.py
Call with python3 -m utils.query
2025-03-07 16:16:34 +01:00
Franciszek Stachura
6a071ff590 utils/speedtest: Make compatible with code changes 2025-03-07 16:16:34 +01:00
Franciszek Stachura
5563feaf78 query: Move dts commands to functions 2025-03-07 16:16:34 +01:00
Franciszek Stachura
ac5a64b972 query: Move 'exists' command to a function 2025-03-07 16:16:34 +01:00
Franciszek Stachura
2a93ee95fb query: Move 'dir' command to a function 2025-03-07 16:16:34 +01:00
Franciszek Stachura
5f9c6236a6 web: Replace query("family") with getFileFamily 2025-03-07 16:16:34 +01:00
Franciszek Stachura
53b1ffe208 query: Move 'file' command to a function 2025-03-07 16:16:34 +01:00
Franciszek Stachura
9bc76f0d8d query: Move 'type' command to a function 2025-03-07 16:16:34 +01:00
Franciszek Stachura
b646d650d4 query: Move 'versions' command to a function 2025-03-07 16:16:34 +01:00
Franciszek Stachura
2440d49731 query: Move 'ident' command to a function 2025-03-07 16:16:34 +01:00
Franciszek Stachura
64efb3d678 query: Move 'latest' command to a function 2025-03-07 16:16:34 +01:00
Théo Lebrun
0e846ccd6e script.sh: parse-defs: C: index defs that have a CONFIG_ in their line
Take the pm_ptr() macro as example:

⟩ ctags -x --kinds-c=+p+x --extras='-{anonymous}' include/linux/pm.h | grep pm_ptr
pm_ptr           macro       475 include/linux/pm.h #define pm_ptr(_ptr) PTR_IF(IS_ENABLED(CONFIG_PM), (_ptr))

The previous grep would remove the pm_ptr() line because it contains
CONFIG_. We only want to remove a line if it starts with CONFIG_.

Testing this commit:

⟩ export LXR_REPO_DIR=...
⟩ hash=97b0e23363c8283f53c361d8129ef90d7a2b9350
⟩ ./script.sh parse-defs $hash pm.h C | sort > /tmp/defs-pm-before.txt
⟩ # TODO: apply commit
⟩ ./script.sh parse-defs $hash pm.h C | sort > /tmp/defs-pm-after.txt
⟩ comm -13 /tmp/defs-pm-before.txt /tmp/defs-pm-after.txt
pm_ptr macro 475
pm_sleep_ptr macro 476

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-02-25 09:28:52 +01:00
Franciszek Stachura
fc8106d520 web: Unify _link and _url variable suffixes 2025-02-14 12:50:29 +01:00
Franciszek Stachura
0f0695e09c web: Stop quoting query('latest') multiple times 2025-02-14 12:49:20 +01:00
Franciszek Stachura
531529a182 web: Make "Go to home page" link to current version
If project and version is valid. Otherwise, redirect to latest or /.
2025-02-14 12:49:16 +01:00
Franciszek Stachura
6edaafb2f3 web: Handle project/version details in invalid URL
Examples:
/musl/v1.2.5/sourc
/musl/v1.2.5/sourc/
/musl/v1.2.5/sourc/asd
/musl/v1.2.5/sourc/asd/
/musl/v1.2.5/sourc/asd/zxc
/musl/v1.2.5/sourc/asd/zxc/
/musl/v1.2.5/sourc/asd/zxc/...

Falcon requires field names on one level to be the same on all
(matching?) paths.
2025-02-14 12:48:20 +01:00
Franciszek Stachura
f6b2d46a0c web: Redirect on incomplete URLs
Examples:
/linux          -> /linux/{latest_tag}/source
/linux/{tag}    -> /linux/{tag}/source
2025-02-14 12:48:20 +01:00
Théo Lebrun
e8823db052 query: query('latest'): return string 2025-02-14 12:48:11 +01:00
Franciszek Stachura
7d09affd5b Dockerfile: Build and use ctags v6.1.0 (upgrade from v5.9)
Changelogs: https://github.com/universal-ctags/ctags/releases

There are some function definitions that are not found by ctags v5.9.
We must upgrade to the latest release but it isn't packaged by Debian.

We only copy the ctags executable. Debian's package files are:
https://packages.debian.org/sid/amd64/universal-ctags/filelist
We drop:
 - The optscript and readtags executables,
 - The changelog and copyright files and,
 - All manpages.
2025-02-14 09:58:00 +01:00
Franciszek Stachura
6559c597c8 web: Update Jinja to v3.1.5
Changelog: https://jinja.palletsprojects.com/en/stable/changes/
2025-02-14 09:51:11 +01:00
Franciszek Stachura
60213dddfa web: Update Falcon from dev to v4.0.2
Changelog: https://falcon.readthedocs.io/en/stable/changes/4.0.0.html

It mentions PathConverter, which was the feature we depended in the
development branch.
2025-02-14 09:50:34 +01:00
Théo Lebrun
5a225c5c43 Dockerfile: move COPY . /usr/local/elixir/ to be the last step
On my machine, a `docker build` after any file change is:
 - Before this patch: 7.29s
 - Afterwards:        2.18s

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-02-05 14:51:30 +01:00
Théo Lebrun
29cf6a5f48 elixir/lib: shorten getDataDir() & getRepoDir()
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-02-03 11:51:32 +01:00
Théo Lebrun
0d2f11baba elixir/lib: fix error case of getDataDir() & getRepoDir()
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-02-03 11:50:53 +01:00
Chris White
973d00665e find-file-doc-comments.pl: don't look for a doc comment before the file starts
Related to #232
2025-01-31 20:25:02 +01:00
Chris White
f9267417fa find-file-doc-comments.pl: die on non-existent input files [minor] 2025-01-31 20:25:02 +01:00
Franciszek Stachura
2cf83f5f7b web: Add input reset button
I tried type="reset", but it reverts to initial value that is sometimes
set by the backend, instead of actually cleraing data.

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/reset
2025-01-31 20:11:53 +01:00
Franciszek Stachura
de31767f6b web: Add types 2025-01-31 19:55:56 +01:00
Théo Lebrun
1921e05ab1 gitignore: add /data/ for use as data root
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:49 +01:00
Théo Lebrun
808ed15dbc Dockerfile: add utils/ in $PATH by default, for easy indexing
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
43a0eba20a Dockerfile: set PYTHONUNBUFFERED=1 by default
This changes the stdout/stderr buffering behavior of Python. Without it,
indexing scripts don't stream updates and use really big buffers.

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
7093c4ab6e Dockerfile: add virtualenv to $PATH by default
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
bb7131fafb utils/index: avoid passing argument to test(1)
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
894920a4ff utils/index: force use of bash, we depend on it for ${@:5} syntax
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
ab553d2e63 README: update following utils/* script changes
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
ebb06050aa utils/index: remove /usr/local/elixir/update.py absolute path
Instead, start from $0 and move back up two times. So, something like:

    ./elixir/utils/index
    ./elixir/utils
    ./elixir
    ./elixir/update.py

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
7f9562a13f utils/index: remove git config --system --add safe.directory call
Stop writing a global file when initializing projects. This can cause
permission issues. We instead pass the option manually for each Git
process call using:

    git -c safe.directory=...

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
d5ffe69f05 utils/index: allow indexing project with remote URLs
Previously, to start an indexing from scratch:

    ./utils/index /srv/elixir-data musl https://git.musl-libc.org/git/musl

This is annoying as the script already has the remote URLs for all known
projects. Now, a call without remote will automatically add the remote
URLs matching the project name:

    ./utils/index /srv/elixir-data musl

This copies the behavior that was previously only implemented for --all.

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
0e9954b702 README: remove "Keeping git repository disk usage under control" section
New script utils/index does an automatic call to `git gc --auto` and if
it detects a gc.log file, it runs `git gc --aggressive`.

There shouldn't be any reason for people to have to think about that
aspect. Remove that info from the README and make it lighter weight.

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
f17c56dc09 utils: deduplicate pack-repositories into index
utils/pack-repositories did the following on repos which have a gc.log
file existing (created when GC fails):

    git prune
    git gc --aggressive
    git prune
    git gc --aggressive

Here we:

 - Delete utils/pack-repositories; we don't want that detection to be
   done manually. Instead, we integrate the gc.log detection into
   utils/index that should be called often.

 - Create a hidden flag ($ELIXIR_GC) to allow manual trigger.

 - Replace the above sequence with a simpler `git gc --aggressive`.
   Let's trust Git.

 - Do a `git gc --auto` in the default case. This call is automatically
   done by porcelain commands but we don't run any so let's give Git an
   opportunity to cleanup from time to time (heuristic based).

 - Replace the gc.log detection from:
      find . -name gc.log
   To:
      test -e $data/$project/repo/gc.log
   It should be more reliable. With the first approach we risk projects
   that contain a file gc.log to trigger the detection on each run.

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
dd4bc063b7 utils/index: add init.defaultBranch= config to git init call
Avoid the following Git warning:

hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
cd89e643fb utils: deduplicate utils/update-elixir-data into utils/index
Previously:

    LXR_PROJ_DIR=/srv/elixir-data ./utils/update-elixir-data

Now:

    ./utils/index /srv/elixir-data --all

The impact is slightly different: it also has the side-effect of
creating all known projects (Linux, U-Boot, etc.) if they didn't exist.
We have asked around and we are not aware of any other Elixir instance.
To keep the previous behavior, if people don't want to index all
supported projects:

    x=/srv/elixir-data
    find $x -mindepth 1 -maxdepth 1 -printf "%f\n | \
        xargs -L1 -r ./utils/index $x

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
359c76f720 utils/index: make it possible to update a specific project
Allow calling like:

    ./utils/index musl

That will do the same thing as before (fetch+index).
It works only if a previous call was made to add remotes.

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
a76c95546d utils: deduplicate index-all-repositories into index
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00
Théo Lebrun
5420995681 utils: rename index-repository to index
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
2025-01-31 19:27:17 +01:00