Month: August 2017

Twitter mapping on Elasticsearch index

Create new mapping for Time in ES index of tweets:

curl -XPUT "http://localhost:9200/twitter_new3" -H 'Content-Type: application/json' -d'
{
 "index.mapping.total_fields.limit": 2000,
 "mappings" : {
 "tweet" : {
 "dynamic" : "false",
 "_all" : {
 "enabled" : false
 },
 "properties" : {
 "postDate" : {
 "type" : "date"
 },
 "geoip" : {
 "properties" : {
 "city_name" : {
 "type" : "keyword"
 },
 "latitude" : {
 "type" : "double"
 },
 "location" : {
 "type" : "geo_point"
 },
 "longitude" : {
 "type" : "double"
 }
 }
 }
 }
 }
 }
}'

 

Kibana query samples

Base url: http://localhost:9200

List all

GET _search
{
  "query": {
    "match_all": {}
  }
}

 

List all indexes

get _cat/indices

Delete indice “andro_sensor_rrr
DELETE andro_sensor_rrr

 

Search for keyword
List all documents

GET twitter_new/_search

List all documents and display only text attribute containing query string

GET twitter_new/_search
{
  "query": {
    "query_string": {
      "default_field": "text",
      "query": "api"
    }
  }
}
GET twitter_new/_search
{
  "_source": {
    "includes": [
      "text"
    ]
  }
}

 

Add mapping

PUT andro_sensor
{
  "mappings": {
    "my_type": {
      "_timestamp": {
        "enabled": true
      }
    }
  }
}

Complex search show only attributes listed in includes, order by created_at field and _score value in descending order.

GET twitter_new/_search?q=IoT
{
  "_source": {
    "includes": [
      "text",
      "created_at",
      "entities.urls.url"
    ]
  },
  "sort": [
    {
      "created_at": {
        "order": "desc"
      }
    },
    {
      "_score": {
        "order": "desc"
      }
    }
  ]
}

Comples search as cURL

POST /twitter_new/_search?q=kaczor AND PIS HTTP/1.1
Host: localhost:9200
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: 65b9524f-4412-1d6d-7a1b-cff6971b7c84

{
  "_source": {
    "includes": [
      "text",
      "created_at",
      "entities.urls.url"
    ]
  },
  "sort": [
    {
      "created_at": {
        "order": "desc"
      }
    },
    {
      "_score": {
        "order": "desc"
      }
    }
  ]
}