index: clearance of old data
Closedopened 1 year ago by René Wagner · 0 comments
old data is not removed from the data store/index once it has been added.
data to be cleaned
- pages which have no successfull crawl since 1 month (last_crawl_success older than 1 month)
delete from page where last_crawl_success_at < (datetime.utcnow() - 1 month) and last_crawl_at => last_crawl_success_at
- pages that where excluded from the crawl
René Wagner added the
questionlabel 1 year ago
René Wagner added
questionlabels 12 months ago
René Wagner changed title from
clearance of old data to index: clearance of old data 10 months ago
René Wagner referenced this issue from a commit 10 months ago
René Wagner self-assigned this 10 months ago
Reference in new issue
There is no content yet.
Delete Branch '%!s(MISSING)'
Deleting a branch is permanent. It CANNOT be undone. Continue?