Mgiyuk

Logstash aggregated error I'm trying to aggregated some results from my statement and i was following this example over here: https://www.javacodegeeks.com/2017/10/aggregate-index-data-elasticsearch-using-logstash-jdbc.html But I'm getting some error's Where's the logstash config # file: simple-out.conf input { jdbc { jdbc_connection_string => "jdbc:mysql://localhost:3306/mestradods" jdbc_user => "root" jdbc_password => "" jdbc_validate_connection => true jdbc_driver_library => "C:Program Files (x86)MySQLConnector.J 5.1mysql-connector-java-5.1.41-bin.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" statement => "SELECT p.*, f.* FROM mestradods.projetos p join mestradods.fileuploaded f on p.projeto_id = f.projeto_id where p.projeto_id > :sql_last_value order by p.projeto_id;" use_column_value => true ...

Elasticsearch 6: Rejecting mapping update as the final mapping would have more than 1 type I'm trying to convert a project to use the latest Elasticsearch 6 and am having this problem. I don't know if the problem is "Product" vs "product". In my mappings and attributes I specify "products", so I am not sure why I get this error when I try to index a product. Error: Invalid NEST response built from a unsuccessful low level call on PUT: /products/products/1?pretty=true&error_trace=true "Rejecting mapping update to [products] as the final mapping would have more than 1 type: [Product, products]" Request: { "id": 1, "warehouseId": 0, "productStatus": 1, "sku": "102377", "name": "Name", "shortDescription": "Description", "longDescription": "Description", "price": 37.3200 } My code: [ElasticsearchTy...

How to move Cloudwatch logs across regions? I'm currently exploring an AWS setup using Lambda@Edge to capture metrics from viewer requests. Since I cannot control where the logs will end up (it simple ends up in Cloudwatch in the region closest to the request origin) I somehow need to transfer logs between regions. I currently have an ElasticSearch cluster in eu-west-1 where I would ultimately want to ingest all my logs, from all regions. What is the recommended approach to move logs in CloudWatch from one region into and ElasticSearch cluster in another region? 1 Answer 1 I think you have two options. You can stream all of your log groups to lambda and have lambda post to ElasticSearch (repeat for every region). You can stream your log groups to ElasticSearch as if it were in another account (repeat for every region). This would require a public endpoint for your ElasticSearch cluster. ...

Conversion of elasticsearch aggregation into code using java REST API is giving error This is working fine in kibana console without any error POST rangedattr_4_1/_search { "size": 0, "aggs": { "user_field": { "terms": { "field": "FRONTLINK_OBJECT_GUID", "size": 10 }, "aggs": { "group_members": { "max": { "field": "MERGED_LINKS_TYPE" } }, "groupmember_bucket_filter": { "bucket_selector": { "buckets_path": { "groupmembers": "group_members" }, "script": "params.groupmembers %2==1" } }, "aggs":{ "top_hits": { "size": 1 } } } } } } But when I converted this into java code, like this: Map<String, String> bucketsPath...

Term, nested documents and must_not query incompatible in ElasticSearch? I have trouble combining term, must_not queries on nested documents. Sense example can be found here : http://sense.qbox.io/gist/be436a1ffa01e4630a964f48b2d5b3a1ef5fa176 Here my mapping : { "mappings": { "docs" : { "properties": { "tags" : { "type": "nested", "properties" : { "type": { "type": "string", "index": "not_analyzed" } } }, "label" : { "type": "string" } } } } } with two documents in this index : { "tags" : [ {"type" : "POST"},...

How to find the distinct values in the array in all the indexes using elasticsearch-dsl? I am using elasticsearch-dsl in django. And I have a DocType document defined and a keyword containing a list of values. Here is my code for the same. from elasticsearch_dsl import DocType, Text, Keyword class ProductIndex(DocType): """ Index for products """ id = Keyword() slug = Keyword() name = Text() filter_list = Keyword() filter_list is the array here which contains multiple values. Now I have some values say sample_filter_list which are the distinct values from and some of these elements can be present in some product's filter_list array. So given this sample_filter_list, I want all the unique elements of filter_list of all the products whose filter_list intersection with sample_filter_list in not null. for example I have 5 products whose filter_list is like : 1) ['a', 'b', 'c'] 2) ['d', 'e'...

How to search for separate key and value fields in an array in ElasticSearch? My ElasticSearch documents contain a nested collection of form fields. Each field has a name and a value and the mapping is as follows: form: { properties: { id: { type: 'integer' }, name: { type: 'text' }, form_data: { type: 'nested', properties: { 'name': { type: 'keyword' }, 'value': { type: 'text', analyzer: 'full_text_analyzer' } } } } } I need to allow the user to search for multiple form fields to refine their search. They can choose which fields to search by and assign a value to each. For example applicant_name = 'Joe' pet_type = 'dog' This would find all documents that contained a field named applicant_name which had a value fuzzy matching Joe as well as a field named pet_type and a value fuzzy matching dog . applicant_name Joe pet_type dog The que...

Elastic Search missing some documents while creating index from another index I have created one index in elastic search name as documents_local and load data from Oracle Database via logstash. This index containing following values. documents_local "FilePath" : "Path of the file", "FileName" : "filename.pdf", "Language" : "Language_name" Then, i want to index those file contents also, so i have created one more index in elastic search named as document_attachment and using the following Java code, i fetched FilePath & FileName from the index documents_local with the help of the filepath i retrived file will is available in my local drive and i have index those file contents using ingest-attachment plugin processor. document_attachment FilePath FileName documents_local filepath Please find my java code below, where am indexing the files. private final static String INDEX = "documents_local"; //Documents...