Batch insert data into Elasticsearch
When I want to batch add documents to elasticsearch, this is how I do
I create an index:
I the perform the insertion:
For example, this will lead to ERROR, because you put "\n" where Elasticsearch dont expect:
When I want to batch add documents to elasticsearch, this is how I do
I create an index:
curl -XPUT localhost:9200/tmp2I prepare the file for batch insertion (official documentation)
{ "create" : { "_index" : "tmp2", "_id" : "1" } }
{"title": "RxJS: How to Use refCount","link": "https://blog.angularindepth.com/rxjs-how-to-use-refcount-73a0c6619a4e","text": "My previous article — Understanding the publish and share Operators — looked only briefly at the refCount method. Let’s look at it more closely here."}
{ "create" : { "_index" : "tmp2", "_id" : "12" } }
{"title": "I reverse-engineered Zones (zone.js) and here is what I’ve found","link": "https://blog.angularindepth.com/i-reverse-engineered-zones-zone-js-and-here-is-what-ive-found-1f48dc87659b","text": "Zones is a new mechanism that helps developers work with multiple logically-connected async operations. Zones work by associating each async operation with a zone."}
Save the file under the name "news.json"I the perform the insertion:
curl -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@news.json"Pay attention to the fact that data must be one line: do not format you JSON data.
For example, this will lead to ERROR, because you put "\n" where Elasticsearch dont expect:
{ "create" : { "_index" : "tmp2", "_id" : "1" } }
{
"title": "RxJS: How to Use refCount",
"link": "https://blog.angularindepth.com/rxjs-how-to-use-refcount-73a0c6619a4e",
"text": "My previous article — Understanding the publish and share Operators — looked only briefly at the refCount method. Let’s look at it more closely here."
}
{ "create" : { "_index" : "tmp2", "_id" : "12" } }
{
"title": "I reverse-engineered Zones (zone.js) and here is what I’ve found",
"link": "https://blog.angularindepth.com/i-reverse-engineered-zones-zone-js-and-here-is-what-ive-found-1f48dc87659b",
"text": "Zones is a new mechanism that helps developers work with multiple logically-connected async operations. Zones work by associating each async operation with a zone."}
The ERROR will be:{"error": { "root_cause":[ { "type":"illegal_argument_exception", "reason":"Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]" }], "type":"illegal_argument_exception","reason":"Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]" }, "status":400 }