Commit Graph

371 Commits

Author SHA1 Message Date
Alessandro Di Federico fd216ef374 daemon: improve logging 2025-10-28 11:43:22 +01:00
Giacomo Vercesi 6abba05d7e Introduce ImportFiles pipe
Introduce the `ImportFiles` pipe which leverages the newly-introduced
`FileStorage` to fetch the required files and store them in the
`BinariesContainer`.
2025-10-16 17:48:45 +02:00
Giacomo Vercesi b622fe8f19 Implement FileProvider interface
Implement the `FileProvider` interface, this allows pipes to request
files given a `FileRequest` (hash and optional size/name).
2025-10-16 17:48:45 +02:00
Giacomo Vercesi 40da49548b Add Binaries list to the model
Add to the model an entry detailing the list of input binaries under
`Binaries`. This will be referenced by `Segments` when needed.
2025-10-16 17:48:45 +02:00
Giacomo Vercesi f17aa20bd7 model.migrations: implicitly increase version
Instead of delegating to the migration the task of increasing the model
version, do it implicitly as part of the `migrate` function.
2025-10-16 16:08:05 +02:00
Giacomo Vercesi 145bd01279 generate-migrations: remove __init__
Remove the `__init__` function from the template used by the
`revng generate-migrations` command, so far it hasn't been used.
2025-10-16 16:08:05 +02:00
Giacomo Vercesi 0f7dc7047d pypeline.Container: fix signature of (de)serialize
Change the signature of `Container.serialize` and
`Container.deserialize` to use `Buffer` instead of `bytes`, as this
makes the functions compatible with the `revng::pypeline::Buffer`
classes.
2025-10-16 16:08:05 +02:00
Giacomo Vercesi d08c490993 pypeline::Model: expose children function
Expose the `revng::pypeline::Model.children` function to python, as it
was not previously exposed.
2025-10-16 12:51:55 +02:00
Giacomo Vercesi 18f0bd88f9 pypeline::Model: fix serialize
The `serialize` function of `revng::pypeline::Model` erroneously
returned a buffer instead of `bytes`, fix the function so that `bytes`
are returned instead.
2025-10-16 12:51:52 +02:00
Giacomo Vercesi f165f99392 pypeline.Model: make deserialize to static
Change the `Model.deserialize` to a static function, which will return
a new instance of the Model.
2025-10-16 10:30:01 +02:00
Giacomo Vercesi c022c51b12 run_pipe.py: normalize arguments with kwargs
When click calls the command handler functions, dashes are converted to
underscores to work with the variable name limitations of python. When
accessing kwargs, convert the argument name to the underscored version.
2025-10-16 10:30:01 +02:00
Giacomo Vercesi 5c0ef7789e pypeline.storage: return invalidated objects
When calling `StorageProvider.invalidate`, return a dictionary that
conveys to the caller which objects were deleted from storage as a
consequence of the paths supplied to the `invalidate` call.
2025-10-15 17:43:09 +02:00
Giacomo Vercesi 5ea2cbdeaf pypeline.storage: fix invalidation
Fix the invalidation logic in both `InMemoryStorageProvider` and
`Sqlite3StorageProvider`, as it did not account for propagating the
invalidation to both children objects and parent objects.
2025-10-15 17:43:09 +02:00
Giacomo Vercesi d92f4cdca3 ObjectID: introduce byte (de)serialization
Add the `to_bytes` and `from_bytes` to `ObjectID`. This allows
serializing an `ObjectID` instance to raw bytes with a specific binary
format.
2025-10-15 11:58:41 +02:00
Giacomo Vercesi 59a7e0716a pypeline.storage.sqlite3: fix model path column
The column stored read model paths was erroneously called
`model_path_hash`, while the entire model was stored instead of a hash.
Rename the column to just `model_path` to avoid any confusion.
2025-10-15 11:58:41 +02:00
Giacomo Vercesi a1a38cf66d pypeline/cli/utils.py: fix wrong Kind usage
`Kind` objects do not have the `__name__` attribute, use the
`deserialize` function instead.
2025-10-15 11:58:11 +02:00
Giacomo Vercesi 6bdc3c9179 fetch-debuginfo: fix parsing of PDB filename
The PDB filename is up to the first `\x00`, everything else after should
be discarded.
2025-10-15 11:58:11 +02:00
Ivan Krysak 00ea93c99e TTG: move metaschema.yml to share/revng/ 2025-10-13 18:33:10 +03:00
Ivan Krysak 04fdc2028a migrations: do not duplicate the template 2025-10-13 18:33:10 +03:00
Ivan Krysak e56f543b1a Split module.bc.zstd 2025-10-13 10:26:41 +03:00
Ivan Krysak ed53e759c9 Do not use -Y flag
Yaml model are now the _only_ way to store them, no point in still
specifying the old flag for it.
2025-10-13 10:26:41 +03:00
Giacomo Vercesi 6d1fbdca97 Implement infrastructure for PipeRuns
Add the needed infrastructures that allow ephemeral classes (`PipeRun`s)
to be wrapped as a regular pypeline `Pipe` class. These automatically
have their dependencies tracked and propagated to the pypeline.
2025-10-03 17:40:45 +02:00
Lauri Vasama 653d451e6c Remove undef name configuration support
Also adds a model migration removing
Configuration.Naming.UndefinedValuePrefix.
2025-09-29 18:05:55 +03:00
Giacomo Vercesi 688b9fe111 Introduce compatibility with revng-pypeline
Add the necessary machinery to allow `Pipe`s, `Analysis`es,
`Container`s, `Model` and `ObjectID` to be implemented in C++ and used
by Python.
2025-09-17 14:02:40 +02:00
Tommaso Fontana 80eb02c207 Pypeline: merge pypeline 2025-09-10 12:05:15 +02:00
Tommaso Fontana 2778458539 Prepare CMakeLists for pypeline merge
Readme and pyproject for revng and revng_internal to the projects
folders.
Updated CMakeLists.txt so we can specify the README and pyproject
paths.
2025-09-10 12:05:15 +02:00
Tommaso Fontana 680f5cacf2 Apply revng check-conventions --force-format 2025-09-10 12:05:14 +02:00
Giacomo Vercesi 1a515afe07 CLIProject: fix _load_model
Fix the `_load_model` method of `CLIProject`. Before this commit it
erroneously did not call `_set_model` on all codepaths, resulting in
cases where `self.model` was incorrectly missing the `_project` field.
2025-09-08 17:36:12 +02:00
Giacomo Vercesi 62421eeaa2 fetch debuginfo: switch default servers
Switch the default servers used by `fetch debuginfo` to revng-managed
ones, as especially the elfutils.org once has been spotty in the past.
2025-08-08 17:52:57 +02:00
Giacomo Vercesi 9f63c7c476 revng daemon: handle PipelineManager no init
Handle the case in `revng daemon` where the `PipelineManager` fails to
init. In this scenario the server responds to all requests with a 500
(with the exception of `/` and `/status`, which return 200).
2025-07-30 15:43:02 +02:00
Giacomo Vercesi 140a49d088 revng daemon: handle ClientDisconnect exceptions
Gracefully handle the occurrence of a `ClientDisconnect` exception while
processing requests in `revng daemon`.
2025-07-30 14:54:17 +02:00
Giacomo Vercesi 615b2a69f8 synchronizer: avoid using multipart
Change the `TransferConfig` used by `S3Synchronizer._upload_file` to
effectively never use multipart upload.
2025-07-28 12:45:11 +02:00
Giacomo Vercesi f027129399 revng daemon: increase logging
Add additional logs to communicate when an upload happens, when a
produce is run and when an analysis is run.
2025-07-25 16:50:19 +02:00
Ivan Krysak 461c49df25 Model: bump version to 5 and introduce migration 2025-07-23 11:38:52 +02:00
Giacomo Vercesi 29d7f2d8a8 artifacts: allow parsing the recompilable archive
Allow the artifacts machinery to parse, via `pycparser`, the
recompilable archive provided by revng via the
`emit-recompilable-archive` artifact.
2025-07-18 09:30:56 +02:00
Giacomo Vercesi fb21b4aef3 revng.tupletree: drop enum_value_to_index
Drop the `enum_value_to_index` function, as it is unused in the entire
codebase.
2025-07-18 09:30:56 +02:00
Giacomo Vercesi d8ec6b7721 revng.model: add serialize() to all TTG types 2025-07-18 09:30:56 +02:00
Giacomo Vercesi 9665b55182 revng.tupletree: assert same type in diffing
Assert that the types are the same when diffing, instead of silently
returning an empty array.
2025-07-18 09:30:56 +02:00
Giacomo Vercesi 82dccc0bea revng.tupletree: add TypedListDescriptor
Add a class implementing the descriptor protocol to allow TypedList
fields to be assigned a `list` and automatically convert it to the
TypedList class.
2025-07-18 09:30:56 +02:00
Giacomo Vercesi a8f0a844bb revng.project: fix auto-extraction bug
When producing artifacts, if the rank is 0 the only element should be
extracted from the produced dict. Fix a wrong assertion that checked
that the returned dict has 0 keys instead of 1.
2025-07-18 09:30:56 +02:00
Giacomo Vercesi 78e66e0d08 cli: move model import idb in separate dir
Move the two python scripts that compose the `revng model import idb`
command into a separate subdirectory.
2025-07-18 09:30:56 +02:00
Giacomo Vercesi 1e857500fb LLVMArtifact: work without zstd support
Have a fallback method that allows creating an llvm module even if the
LLVM used is not patched to read zstd-compressed modules.
2025-07-18 09:30:56 +02:00
Giacomo Vercesi 69dc55fa9d get_artifact: produce all when using binary
When passing the binary root object, produce all the targets.
2025-07-18 09:30:55 +02:00
Giacomo Vercesi 38b50c2977 tupletree: fix Reference class
Fix the `Reference` class, make it work with the new model and allow it
to be dereferences via the `_accessor` attribute.
2025-07-18 09:30:55 +02:00
Giacomo Vercesi 8065ebf634 Add revng.model.TypeId
Add a class that allows indexing `model.TypeDefinitions` with a single
object.
2025-07-18 09:30:55 +02:00
Giacomo Vercesi f3cccf8bda revng/project: make some attributes private
Rename some attributes to make them private, as they won't be used by
the end-user.
2025-07-18 09:30:55 +02:00
Giacomo Vercesi 460ffbf965 project: move some methods to Binary
Move some methods from the `Project` class to `model.Binary`.
2025-07-18 09:30:55 +02:00
Giacomo Vercesi d22d9201f5 Artifact: replace dump() with raw_data
Replace the `dump()` function with the `raw_data` attribute.
2025-07-18 09:30:55 +02:00
Alessandro Di Federico 4378576eed Python scripting: get_example_binary{,_path} 2025-07-18 09:29:09 +02:00
Giacomo Vercesi 5a05b834ed python: autogenerate module version for wheels
Autogenerate a different version for wheels every time the project is
configured.
2025-07-15 10:46:19 +02:00