This gave suboptimal results in terms of indexing time,
plus a stuck job. Should be investigated further before
changing this.
This reverts commit e1dd2b4881.
Instead of having a complicated thread count scheme and
ending up wasting time waiting for the docs task to finish
with a too low thread count at the end, let's run all
tasks with same number of threads, which by default
is the number of CPUs in the system.
This way, the indexing work always uses as many CPUs
as possible, especially when same tasks are finished
before the others.
The OS shouldn't bother if we try to run more threads
than the actual number of CPUs.
Set the ELIXIR_THREADS environment variable
if you want to use a lower number of threads,
typically on a desktop machine on which you want some
responsiveness.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
As Chris White said it, it's better to have only one line to change if
we have to update that value.
Signed-off-by: Maxime Chretien (MixLeNain) <mchretien@linuxmail.org>
If an ident is in db.defs it's already a valid ident.
So we need to test the idents with lib.isIdent before adding an ident in
db.defs and not while parsing references.
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
This prevent the script from locking at line 578 because UpdateIds will never send an event to wake up the threads (line 91) if there is no tag to add.
Also change len(tag_buf) to num_tags in line 548, no need to compute the
length again as we already have it.
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
We don't need multiple threads for that as it is significantly faster
than defs, refs, docs and comps
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
Idxes needs to be sorted in RefLists but documentation idxes are lower
than others and we added them latter in the RefList so there was a
problem. The easiest solution to solve that is to create a separate
database for documentations.
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
The threads are then distributed among the functions using a specific
rule (see code comments)
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
It's now easier to change the number of threads for each functions
Refs are always what remains at the end so more threads will be helpfull
to finish faster
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
Use 7 threads to do more things at once and spread the load
Lock file before the loop so we don't lock and unlock all the time
This will hopefully reduce database creation time
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
This brings the idea of file families, each ident is identified by his
family and can be referenced in compatible families.
For exemple :
- A Kconfig ident can be referenced in a C file but not in a Devicetree file.
- A Devicetree ident is only referenced in Devicetree files.
- A C ident is only referenced in C files.
Kconfig idents are defined without the CONFIG_ at the beginning.
We add it while indexing for an easier processing of other files.
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
This allows to index multiple things at the same time so it speeds up
database indexation.
Signed-off-by: Maxime Chretien <maxime.chretien@bootlin.com>
- find-file-doc-comments.pl: new file
- data.py: Add database to store doc-comment locations
- script.sh: Add parse-docs subcommand
- update.py:
- Add code to process doc comments
- Update some variable names in hopes of reducing confusion
- query.py:
- Add code to report doc comments
- Update some variable names in hopes of reducing confusion
Also:
- t/TestEnvironment.pm: Add find_doc attribute
- t/interact.pl: Don't die if update.py fails
- t/TestHelpers.pm: Permit checking specific sections of query.py output
- t/300: update regexes per the preceding
- gitignore tags (ctags output) and .cache (api_test.py output)
- For long runs, allows to see which project we are in, and the
percentage of processed tags
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Include C++ files when indexing and tokenizing files.
Unfortunately due to how C++ utilizes namespaces and classes, this
currently introduces a large amount of false positives for unqualified
identifiers that look the same.
Signed-off-by: Chen-Yu Tsai <wens@csie.org>
Python's standard library provides a function to split out file
extensions, and it also handles dot files correctly.
Use that instead of just retrieving the last two characters of
the file name.
This should help with issue #27.
Signed-off-by: Chen-Yu Tsai <wens@csie.org>
