flush the index periodically during rebuild

currently, the indexing can take some time when it is done from scratch on a large repository, and requires a large amount of memory. It would be good to periodically checkpoint the index with a set of artifacts already completed.

How this affects the timestamp used needs to be interpreted, as if the indexing is stopped it may want to avoid reindexing those already done. Perhaps an alternative is to sort the discovered artifacts by timestamp, and setting it to the most recent one done - this can be considered in conjunction with the current issue that synced artifacts sometimes are new but have a timestamp older than the last indexing time and so are not indexed.