You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Gemini Universal Search (GUS)
- Install python and poetry
- Run: "poetry install"
Making an initial index
Make sure you have some gemini URLs for testing which are nicely sandboxed to avoid indexing huge parts of the gemini space.
- Create a "seed-requests.txt" file with you test gemini URLs
- Run: "poetry run crawl -d"
- Run: "poetry run build_index -d"
Now you'll have created
index.new directory, rename it to
Running the frontend
- Run: "poetry run serve"
- Navigate your gemini client to: "gemini://localhost/"
Updating the index
- Run: "poetry run crawl"
- Run: "poetry run build_index"
- Restart frontend
Running test suite
Run: "poetry run python -m pytest"
Roadmap / TODOs
- TODO: improve crawl and build_index automation
- TODO: get crawl to run on a schedule with systemd
- TODO: add functionality to create a mock index
- TODO: exclude raw-text blocks from indexed content
- TODO: strip control characters from logged output like URLs
- TODO: fix bug in calulation of backlinks (iirc the bug is visible on gemini.circumlunar.space)
- TODO: refactor manual exclusion logic to be regex-based instead of prefix-based. we could get more nuanced with exclusion logic this way