Commit graph

918 commits

Author SHA1 Message Date
Franciszek Stachura
4f03a83287 autocomplete: Move to main application 2024-08-27 11:32:19 +02:00
Franciszek Stachura
2a7119fcff autocomplete: Make faster by utilizing DB_SET_RANGE
for prefix search

https://stackoverflow.com/questions/12348346/berkeley-db-partial-match

This *should* return the same results as the original autocomplete.

Steps the original autocomplete takes to find identifiers:
1. It dumps all keys using db.keys() into a file
    query.py:Query.query('keys')
    data.py:BsdDB.get_keys()
    https://pybsddb.sourceforge.net/bsddb3.html - keys(txn=None)
github.com/virtuozzo/cdn-bsddb3-python/blob/fbb1a877/Lib3/bsddb/dbobj.py#L171
github.com/virtuozzo/cdn-bsddb3-python/blob/fbb1a877/Modules/_bsddb.c#L8565
github.com/virtuozzo/cdn-bsddb3-python/blob/fbb1a877/Modules/_bsddb.c#L3730
    seems that this just iterates over the db calling get with DB_NEXT
    on the cursor
    https://docs.oracle.com/cd/E17276_01/html/api_reference/C/dbcget.html
2. Iterates over the keys, looking for keys that start with provided string.
    The script stops when it finds 10 items.

So it's assumed that the order of keys returned by get(DB_NEXT) will make
sense.

This version finds the first key that starts with the prefix using
get(DB_SET_RANGE) and then, just like the previous autocomplete,
iterates over the keys until it finds 10 matching keys.
Now, could get(DB_SET_RANGE) skip keys (ex. point to some other key than
the first key that starts with the prefix)?
I think not - according to the docs, it should point to "the smallest key
greater than or equal to the specified key". The comparison function
determines what "greater or equal" means.
The default comparison function compares keys lexically, with shorter
keys before longer keys.

https://docs.oracle.com/cd/E17276_01/html/api_reference/C/dbset_bt_compare.html

I believe, although I couldn't find precise information about this,
that order of keys when using DB_SET_RANGE shouldn't change.

So tl;dr I'm mostly sure this should work the same way as it worked
before, just faster. There could still be issues with how keys are
ordered in results. The default function seems to just order
characters by byte values.

github.com/berkeleydb/libdb/blob/master/src/btree/bt_compare.c#L154

Also, this allows identifiers sent to autocomplete to contain commas.
2024-08-27 11:32:19 +02:00
Franciszek Stachura
4174b85f2d autocomplete: Move to Falcon 2024-08-27 11:32:19 +02:00
Franciszek Stachura
bcc76c521b web.py: Refactor to use the Falcon framework
* Add requirements.txt with packages used by WSGI Elixir
* Make API compatible with latest Falcon
* Remove remaining global variables from web.py
* Remove Parsed__Path

Falcon passes parsed path segments as arguments to handler methods.
I think it really does not make sense anymore to put that information
back into a tuple just to unpack it again.

* Add RawPathComponent middleware to support encoded slashes

Some paths previously accepted by Elixir can contain encoded slashes.
For example:
/arm-trusted-firmware/sandbox%2Flts-v2.10.3-20240405T0714/source

Falcon by default uses a version of URL that is already decoded. This
default makes parsing said paths impossible.

https://falcon.readthedocs.io/en/v3.1.2/user/recipes/raw-url-path.html

* Move request context building to middleware
* Move validation and unquoting to path converters
* Move base url generation to new functions
2024-08-27 11:32:17 +02:00
Franciszek Stachura
e8da1bf578 Reduce sidebar flashing on navigation
by moving some dynamic HTML generation to templates.
Also fixes no-js CSS a bit by hiding useless elements.
2024-08-20 14:32:52 +02:00
Franciszek Stachura
7d629429bf Generate different meta tag descriptions for different views
Added an ability to customize contents of meta description tag in
templates that inherit from layout. This replaces the default
description previously used in all views.
This change could help with SEO and missing descriptions in search
results (#167). It's based on advice from
https://developers.google.com/search/docs/appearance/snippet

> Create unique descriptions for each page on your site
> Programmatically generate descriptions

This is not guaranteed to fix the issue, search engines are opaque.

All custom descriptions start with "Elixir Cross Referencer - ".

Currently customized descriptions:
* source - path to the file, project name and version
* ident - name of the searched identifier, project name and version

Everything else uses the previous, default description.
2024-08-20 10:35:17 +02:00
Franciszek Stachura
43b4f720b4 Prefer GAS Lexer for .S files
Currently, Pygments often picks SLexer for .S files. It's a lexer for
files related to the R language.

https://pygments.org/docs/lexers/#pygments.lexers.r.SLexer

```
>>> import pygments.lexers
>>> pygments.lexers.guess_lexer_for_filename('arch/x86/boot/header.S',
        open('arch/x86/boot/header.S').read())
<pygments.lexers.SLexer>
```

This commits makes sure Elixir prefers the GAS lexer for .S files
instead of SLexer.
It's not the best heuristic, but better than picking the SLexer class
sometimes.
Currently, Elixir does not index any R language projects. In the future,
it would be better to specify per-project information about what
languages are used.
2024-08-20 10:34:58 +02:00
Franciszek Stachura
0de9a05d1f Increase min-width for tablet mode
Nowadays smartphones are long but narrow. On many devices, the sidebar
is visible (and impossible to hide) in landscape mode.

Long term it would be better to implement sidebar hiding regardless of
device size, but that may require cookies - otherwise some people will
have to hide the sidebar manually on all links.
2024-08-19 17:30:29 +02:00
Franciszek Stachura
538bbeff43 Make dtscompcode filter quit early for some files
files that don't contain assignments to a 'compatible' property
2024-08-09 18:51:39 +02:00
Franciszek Stachura
22296b6cf8 Fix incorrectly issued 301 redirects on latest for project roots
This commit adds a small piece of Javascript code to force browsers to forget
about incorrectly issued 301 redirects from "latest" versions.
This code only runs for root project paths (i.e. `/linux/v6.10.2/source`, but
not `/linux/v6.10.2/source/arch`) - other links with "latest" will remain
broken for people who visited them (at least until they clear their cache).
2024-08-06 12:47:40 +02:00
Théo Lebrun
b1ce7d1654 http/web.py: create constants for HTTP status codes
The goal is to avoid using the wrong status in the wrong location.
"HTTP_STATUS_MOVED_PERMANENTLY" is more explicit than "301".
2024-08-06 11:18:37 +02:00
Franciszek Stachura
8c5ae11e03 Use 302 instead of 301 for redirecting from latest 2024-08-06 11:17:48 +02:00
Franciszek Stachura
22c1d7646d Remove custom ctags version from Dockerfile
As mentioned in 8ffb3d9, required changes were merged into upstream.

Version packaged for Debian 12 seems to contain these changes.
Ubuntu 22.04 mentioned in commit description is based on Debian 12.
universal-ctags has the same version string in both repos.

https://packages.debian.org/bookworm/amd64/universal-ctags
https://packages.ubuntu.com/jammy/universal-ctags
2024-08-05 15:17:23 +02:00
Franciszek Stachura
3714430aef Remove 'HttpProtocolOptions Unsafe' apache.conf insertion from Dockerfile
* Also remove mention of 'HttpProtocolOptions Unsafe' from README
2024-08-05 15:15:57 +02:00
Franciszek Stachura
d6df6deeaa Add information about PYTHONUNBUFFERED env var to README
Without it update logs don't show up on docker run
2024-08-05 15:15:01 +02:00
Franciszek Stachura
ffe8d79c40 Simplify and refactor Dockerfile
* Remove repository indexing from Dockerfile
* Copy Elixir code from the context instead of cloning it from Github
* Move Dockerfile from docker/debian/Dockerfile to docker/Dockerfile
* Add docs for the new Dockerfile
* Add an utility script that allows easier repository configuration
* Add .dockerignore
* Use relative paths in Docker section in README
2024-08-05 14:54:03 +02:00
Michael Opdenacker
662482ebce Audio course course banner
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
2024-08-05 14:20:10 +02:00
Franciszek Stachura
f7341fc285 Remove support for exec filters
* Move configin filter to new filter interface
2024-08-02 15:42:42 +02:00
Franciszek Stachura
3cbbf58414 Move Kconfig, defconfig and make filters to new filter interface 2024-08-02 15:30:26 +02:00
Franciszek Stachura
da18c75560 Move cppinc filters to new filter interface 2024-08-02 15:30:08 +02:00
Franciszek Stachura
9a396cd840 Move DTS filters to new filter interface 2024-08-02 15:30:08 +02:00
Franciszek Stachura
7ecad127b7 Move ident filter to new filter interface, create filter context in web.py 2024-08-02 15:30:08 +02:00
Franciszek Stachura
afdc1c4512 Add Filter interace, class for context used by filters
* Move filter utils to a new file
2024-08-02 15:30:08 +02:00
Franciszek Stachura
f5f49b6b39 Quick fix for searching identifiers with commas
Fixes #309, both manual search and display in search bar.
2024-08-02 13:43:14 +02:00
Franciszek Stachura
06ba355912 Fix extension match for filters
Fixes cppinc, and dts filters
2024-08-01 15:51:44 +02:00
Franciszek Stachura
bd4a7734c1 Change tag to version_unquoted in generate_ident_page 2024-07-31 11:33:53 +02:00
Franciszek Stachura
86af364b14 Generate pretty HTML for more errors 2024-07-31 11:33:53 +02:00
Franciszek Stachura
3d34855677 Make full width of directory entry in tree template clickable 2024-07-31 11:33:49 +02:00
Franciszek Stachura
decc2680b2 Move title generation to templates 2024-07-30 15:16:11 +02:00
Franciszek Stachura
8932f26a22 Move topbar family list to web.py 2024-07-30 15:16:07 +02:00
Franciszek Stachura
ddf00119b8 Minor variable names and arguments changes
* Move get_version lambdas to new variable
2024-07-30 15:12:17 +02:00
Franciszek Stachura
4c22738687 Move ident URL generation to web.py 2024-07-30 15:11:32 +02:00
Franciszek Stachura
5c92b378ea Create file URLs for tree in web.py 2024-07-30 14:10:51 +02:00
Franciszek Stachura
ee4ccdeb80 Provide base source and ident URLs for project and tag as variables 2024-07-30 14:10:51 +02:00
Franciszek Stachura
bf6f183189 Don't display size for directories
* Refactor projects list
2024-07-30 14:10:51 +02:00
Franciszek Stachura
d107fa27aa Remove current_url template variable
Now that the base tag is gone it's not useful anymore

* Make line number links use relative paths
2024-07-30 14:10:51 +02:00
Franciszek Stachura
8cf71f5e62 Remove base tag and baseurl template variable 2024-07-30 14:10:51 +02:00
Franciszek Stachura
21802b3203 Make dynamic references popup use absolute paths 2024-07-30 14:10:51 +02:00
Franciszek Stachura
edce92a64c Make url in templates and web.py absolute
This is a step to removing the base tag. Once everything uses absolute
URLs, base tag will be removed and some URLs will be replaced
by relative URLs.
2024-07-30 14:10:51 +02:00
Franciszek Stachura
5c6e604b9b Generate links for versions sidebar outside of template context
* Replace url variable with current_url that contains the current
  absolute url. This gets us a little closer to getting rid of the base
  tag
2024-07-30 14:10:51 +02:00
Franciszek Stachura
dbc7191060 Change template variable names to be more descriptive 2024-07-30 14:10:51 +02:00
Franciszek Stachura
fabbc52d36 Move filter path exceptions handling filter_applies
Note that this also causes postfunc to not run on files where prefunc was not
applied because filter_applies returned False. This could be a mistake,
although it seems that the only filter that uses path exceptions (cppthinc filter
modified from linux filter) should be compatible with this behavior.
2024-07-30 14:10:51 +02:00
Franciszek Stachura
ba5c4007a2 Move code formatting to format_code 2024-07-30 14:10:47 +02:00
Franciszek Stachura
d1f70679df Remove fake print from web.py
Replace all instances of realprint with print
2024-07-30 12:49:29 +02:00
Franciszek Stachura
554bb11381 Refactor directory tree generation to use a namedtuple
* Fix #302 - Symlinks redirect to invalid paths (but only the directory
  tree part)
* Move more view-related logic to the template
2024-07-30 12:49:29 +02:00
Franciszek Stachura
f08aeb7b76 Split layout into sidebar and topbar 2024-07-30 12:49:29 +02:00
Franciszek Stachura
585d3e3243 Move directory entries generation to a new function
* Remove redundant code parameter in generate_source
2024-07-30 12:49:29 +02:00
Franciszek Stachura
6e0a1ad64c Move source page generation to templates 2024-07-30 12:49:29 +02:00
Franciszek Stachura
36ae9046f4 Move ident template to a new file 2024-07-30 12:49:29 +02:00
Franciszek Stachura
aa5057264d Move ident page HTML generation to layout template 2024-07-30 12:49:29 +02:00