Delete documents from ElasticSearch index in python

Question

Using elasticsearch-py, I would like to remove all documents from a specific index, without removing the index. Given that delete_by_query was moved to a separate plugin, I want to know what is the best way to go about this?

@cricket_007 I could, but I'd rather do it by removing the documents. Otherwise, I'd have to check the index settings and mappings and use them when recreating index. I think it's easier to remove the documents. — zanderle
– zanderle, Commented Feb 18, 2016 at 17:45
A simple backup of the mappings and such shouldn't be that difficult. A full index scan and a bulk delete doesn't seem "easier", IMO — OneCricketeer
– OneCricketeer, Commented Feb 18, 2016 at 18:00

Hkntn · Accepted Answer · 2016-02-21 17:42:03Z

2

It is highly inefficient to delete all the docs by delete by query. More direct and correct action is:

Getting the current mappings (Assuming you are not using index templates)
Dropping the index by DELETE /indexname
Creating the new index and the mappings.

This will take a second, former will take much, much more time and unnecessary disk I/O

answered Feb 21, 2016 at 17:42

Hkntn

3661 silver badge6 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

zanderle Over a year ago

Thank you. It's what I did in the end. I'll leave the other answer as accepted, since it answers the question more directly (even if it is the wrong approach).

Chris Franklin · Accepted Answer · 2016-02-18 17:11:52Z

0

Use a Scroll/Scan API call to gather all Document IDs and then call batch delete on those IDs. This is the recommended replacement for the Delete By Query API based on the official documentation.

EDIT: Requested information for using this specifically in elasticsearch-py. Here is the documentation for the helpers. Use the Scan helper to scan throgh all documents. Use the Bulk helper with the delete action to delete all the ids.

edited Feb 18, 2016 at 17:11

answered Feb 18, 2016 at 16:50

Chris Franklin

4,0322 gold badges18 silver badges20 bronze badges

2 Comments

zanderle Over a year ago

Can you provide more information about how to do this in elasticsearch-py?

Chris Franklin Over a year ago

Added a link to the specific docs for the Python helpers you need to perform the scan and then call the bulk deletion. Everything you need should be there!

Collectives™ on Stack Overflow

Delete documents from ElasticSearch index in python

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related