Skip to main content

elasticsearch bulk add

Batch insert data into Elasticsearch
When I want to batch add documents to elasticsearch, this is how I do
I create an index:
curl -XPUT localhost:9200/tmp2
I prepare the file for batch insertion (official documentation)

{ "create" : { "_index" : "tmp2", "_id" : "1" } }
{"title": "RxJS: How to Use refCount","link": "https://blog.angularindepth.com/rxjs-how-to-use-refcount-73a0c6619a4e","text": "My previous article — Understanding the publish and share Operators — looked only briefly at the refCount method. Let’s look at it more closely here."}
{ "create" : { "_index" : "tmp2", "_id" : "12" } }
{"title": "I reverse-engineered Zones (zone.js) and here is what I’ve found","link": "https://blog.angularindepth.com/i-reverse-engineered-zones-zone-js-and-here-is-what-ive-found-1f48dc87659b","text": "Zones is a new mechanism that helps developers work with multiple logically-connected async operations. Zones work by associating each async operation with a zone."}
Save the file under the name "news.json"
I the perform the insertion:
curl -H "Content-Type: application/x-ndjson" -XPOST  localhost:9200/_bulk --data-binary "@news.json"
Pay attention to the fact that data must be one line: do not format you JSON data.
For example, this will lead to ERROR, because you put "\n" where Elasticsearch dont expect:

{ "create" : { "_index" : "tmp2", "_id" : "1" } }
{
 "title": "RxJS: How to Use refCount",
 "link": "https://blog.angularindepth.com/rxjs-how-to-use-refcount-73a0c6619a4e",
 "text": "My previous article — Understanding the publish and share Operators — looked only briefly at the refCount method. Let’s look at it more closely here."
}
{ "create" : { "_index" : "tmp2", "_id" : "12" } }
{
 "title": "I reverse-engineered Zones (zone.js) and here is what I’ve found",
 "link": "https://blog.angularindepth.com/i-reverse-engineered-zones-zone-js-and-here-is-what-ive-found-1f48dc87659b",
 "text": "Zones is a new mechanism that helps developers work with multiple logically-connected async operations. Zones work by associating each async operation with a zone."}
The ERROR will be:
{"error":
 {
  "root_cause":[
   {
    "type":"illegal_argument_exception",
    "reason":"Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]"
   }],
  "type":"illegal_argument_exception","reason":"Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]"
 },
  "status":400
}


Popular posts from this blog

npm run build base-href

Using NPM to specify base-href When building an Angular application, people usually use "ng" and pass arguments to that invocation. Typically, when wanting to hard code "base-href" in "index.html", one will issue: ng build --base-href='https://ngx.rktmb.org/foo' I used to build my angular apps through Bamboo or Jenkins and they have a "npm" plugin. I got the habit to build the application with "npm run build" before deploying it. But the development team once asked me to set the "--base-href='https://ngx.rktmb.org/foo'" parameter. npm run build --base-href='https://ngx.rktmb.org/foo did not set the base href in indext.html After looking for a while, I found https://github.com/angular/angular-cli/issues/13560 where it says: You need to use −− to pass arguments to npm scripts. This did the job! The command to issue is then: npm run build -- --base-href='https://ngx.rktmb.org/foo&

Emacs Pulumi LSP

Install Pulumi Emacs Mode The source code is on https://github.com/pulumi/pulumi-lsp. At the very bottom of the page are the instructions on how to install. You Need to have make installed. $ make install emacs-client mkdir -p ./bin go build -ldflags "-X github.com/pulumi/pulumi-lsp/sdk/version.Version=v0.2.3-6-gec49054" -o ./bin -p 10 ./cmd/... go: downloading github.com/pulumi/pulumi/sdk/v3 v3.53.1 ... ... ... go install -ldflags "-X github.com/pulumi/pulumi-lsp/sdk/version.Version=v0.2.3-6-gec49054" ./cmd/... mkdir -p editors/emacs/bin cd editors/emacs && emacs -Q --batch --eval "(progn (setq package-user-dir \"$(pwd)/bin\" \ package-archives '((\"melpa\" . \"https://melpa.org/packages/\") \

emacs29 intelephense

Emacs 29 and PHP Intelephense I use to use Emacs and PHP Intelephense for PHP development. I recently upgraded to Emacs 29 and PHP Intelephense stopped working. I found a solution on Reddit Based on that, I rewrote my .emacs file to use eglot instead of lsp-mode, and this is the result. (use-package eglot :ensure t) (add-hook 'php-mode-hook 'eglot-ensure) (use-package php-mode :ensure t :mode ("\\.php\\'" . php-mode)) (add-to-list 'auto-mode-alist '("\\.php$" . php-mode)) (provide 'lang-php) (use-package company :ensure t :config (setq company-idle-delay 0.3) (global-company-mode 1) (global-set-key (kbd "M- ") 'company-complete)) (require 'eglot) (add-to-list 'eglot-server-programs '((php-mode :language-id "php") . ("intelephense" "--stdio" :initializationOptions (:licenseKey "98989898989898989898"