sidx/README.md

[sidx](https://forge.someonex.net/else/sidx)
===


Work in Progress.
Indexing archives and build outputs.


Roadmap
---


- [x] PoC: basic `reqwest` fetcher, minimal `snix-castore` integration, claims of the form "`$uri` referred to `$blake3` at `$time`" stored in sqlite.
- [ ] Zip, tar, git fetchers.
- [ ] ELF scanning: claims of the form "`$blake3` is ELF" and "`$blake3`'s `DT_RUNPATH` is `$runpath`", etc. (`DT_NEEDED`, `.interp`, symbols exported in `.data` and `.text`)
- [ ] `cuobjdump` scanner: claims of the form "`$blake3` contains `PTX` or `SASS` code for `$gencode`".

Approach
---

Vapourware and means to an end:
[this project](https://forge.someonex.net/else/sidx) was originally motivated by the needs of maintaining
`cudaPackages` in Nixpkgs.
Specifically, it attempts to answer the question of "what is there to be maintained",
improve [observability and debug-ability of the package set (cf. demo)](https://cuda-index.someonex.net/sidx/UriReference).
cuda: support cudnn manifests Also changes the schemas a bit (still don't care about migrations), switches to named_params!, fixes up perf issues when ingesting manifests and/or hrefs into sqlite, etc. Adds sample queries such as "CudaArtifact conflicts" to datasette config, which explain some issues associated with choosing an evalModules schema on the cudaPackages side. 2025-05-09 04:01:38 +00:00			`[sidx](https://forge.someonex.net/else/sidx)`
init First attempt at both Rust and snix-castore. I'll start by putting together a single-file CLI app, no splitting modules out until needed, no separation of concerns. Currently, just the fetching is implemented: ``` $ sidx pkgs/dev/cuda-/cuda/manifests/*.json ``` npins: init 2025-04-15 14:07:49 +00:00			`===`


cuda: support cudnn manifests Also changes the schemas a bit (still don't care about migrations), switches to named_params!, fixes up perf issues when ingesting manifests and/or hrefs into sqlite, etc. Adds sample queries such as "CudaArtifact conflicts" to datasette config, which explain some issues associated with choosing an evalModules schema on the cudaPackages side. 2025-05-09 04:01:38 +00:00			`Work in Progress.`
init First attempt at both Rust and snix-castore. I'll start by putting together a single-file CLI app, no splitting modules out until needed, no separation of concerns. Currently, just the fetching is implemented: ``` $ sidx pkgs/dev/cuda-/cuda/manifests/*.json ``` npins: init 2025-04-15 14:07:49 +00:00			`Indexing archives and build outputs.`


			`Roadmap`
			`---`


			- [x] PoC: basic `reqwest` fetcher, minimal `snix-castore` integration, claims of the form "`$uri` referred to `$blake3` at `$time`" stored in sqlite.
			`- [ ] Zip, tar, git fetchers.`
			- [ ] ELF scanning: claims of the form "`$blake3` is ELF" and "`$blake3`'s `DT_RUNPATH` is `$runpath`", etc. (`DT_NEEDED`, `.interp`, symbols exported in `.data` and `.text`)
			- [ ] `cuobjdump` scanner: claims of the form "`$blake3` contains `PTX` or `SASS` code for `$gencode`".

			`Approach`
			`---`

cuda: support cudnn manifests Also changes the schemas a bit (still don't care about migrations), switches to named_params!, fixes up perf issues when ingesting manifests and/or hrefs into sqlite, etc. Adds sample queries such as "CudaArtifact conflicts" to datasette config, which explain some issues associated with choosing an evalModules schema on the cudaPackages side. 2025-05-09 04:01:38 +00:00			`Vapourware and means to an end:`
			`[this project](https://forge.someonex.net/else/sidx) was originally motivated by the needs of maintaining`
			`cudaPackages` in Nixpkgs.
			`Specifically, it attempts to answer the question of "what is there to be maintained",`
			`improve [observability and debug-ability of the package set (cf. demo)](https://cuda-index.someonex.net/sidx/UriReference).`