308 Commits (master)
 

Author SHA1 Message Date
René Wagner acd728e7c4 update 2021-06-04 1 week ago
René Wagner d3b1dd8e77 more exception handling on link update 2 weeks ago
René Wagner 3f7c0f84f9 fix wrong embedding of excludes 2 weeks ago
René Wagner 8b004af54d unify capitalisation of charset in statistics 2 weeks ago
René Wagner 5c9e5267cf move exclude definition to own file 3 weeks ago
René Wagner 14c3997724 news 2021-05-25 3 weeks ago
René Wagner e0fba80405 some exception handling and updated service files 3 weeks ago
René Wagner 52d2b4c86d fix last wrong exception in crawl 4 weeks ago
René Wagner 9b6ef8a0e2 fix wrong exception handling in crawl 4 weeks ago
René Wagner 06c0258323 update 2021-05-12 1 month ago
René Wagner 9b21f64790 rewrite statistics gathering to pure sql 1 month ago
René Wagner 1266d9a93b exception handling on page save 1 month ago
René Wagner 20b0924233 news 2021-04-14 2 months ago
René Wagner f6c3526288 delete tmp files of whoosh 2 months ago
René Wagner 61d713038c use .fromisoformat for getting timestamp from db 3 months ago
René Wagner e6faa0e129 various corrections 3 months ago
René Wagner f5ce631246 hack: index update in separate dir 3 months ago
René Wagner 0b0b33610a skip a capsule after 5 consecutive failed requests 3 months ago
René Wagner 1dac97f01e workaround for "index update blocks searches" 3 months ago
René Wagner 2ebc1a844a news update 2021-03-08 3 months ago
René Wagner 6e54e52014 Merge branch 'master' of git://natpen.net/gus 3 months ago
René Wagner c791a758f2 update poetry deps 3 months ago
René Wagner e691231ec8 gsi specific updates 2021-02-26 4 months ago
René Wagner 8520ec533c robots.txt sections "*" and "indexer" are honored 4 months ago
René Wagner 134b7f6c48 correctly handle robots.txt 4 months ago
René Wagner 64748f0852 add verbose search to robots.txt 4 months ago
René Wagner 4817ab3149 add verbose search to robots.txt 4 months ago
René Wagner 108bfe850a correctly handle robots.txt 4 months ago
René Wagner 95e29af321 Merge branch 'master' of git://natpen.net/gus 4 months ago
René Wagner 569baa0637 limit max_crawl_depth to 100 for normal crawl 4 months ago
René Wagner 42af8b76b3 increase frequency to avoid rescanning within a single crawl 4 months ago
René Wagner af967cc728 add some forbidden URIs & set max_crawl_depth 4 months ago
René Wagner 39edf72847 remove seed-requests from repo 4 months ago
René Wagner 105f1ca2c6 Merge branch 'master' of git://natpen.net/gus 4 months ago
Natalie Pendragon df9486f3ef Add a few more url parsing test cases 4 months ago
Natalie Pendragon 788291199d Update to Python 3.9 compatibility 4 months ago
René Wagner 39ca213bb5 update python deps 4 months ago
René Wagner 44812ee8b5 introduce systemd-unit for indexer 4 months ago
René Wagner e897bc488b update python deps 4 months ago
René Wagner e1f673fc7d updates geminispace.info 2021-02-02 4 months ago
René Wagner b119e8b6d8 introduce systemd-unit for indexer 4 months ago
René Wagner 4edb3fc7d4 gsi specific updates 4 months ago
Natalie Pendragon bb377d6f0a Make README heading lines more consistent 4 months ago
Natalie Pendragon 3908a24b94 Fix trailing whitespace and reformat long string 4 months ago
Natalie Pendragon d9fc0b3d0b Make README heading lines more consistent 4 months ago
René Wagner d4e7acc7ae add systemd-units for automatic crawling 4 months ago
Natalie Pendragon 29d6013770 Fix trailing whitespace and reformat long string 4 months ago
René Wagner 7b37090a8e add "/robots.txt" route to views.py 5 months ago
René Wagner 2c9c66392b gsi specific updates 2021-01-29 4 months ago
René Wagner 6396d9f186 add systemd-units for automatic crawling 4 months ago