标签：index gt 10 parsing json 2021 Error logstash

1、生产环境有部分报错

信息如下：

2021-10-15T15:49:28,932][WARN ][logstash.filters.json    ][main][7e17a3dc7e2c6df08ed7012ca6bfe17e3277d05d745f2c5bf55d2b01b151e25b] Error parsing json {:source=>"message", :raw=>"查询用户详情接口", :exception=>#<LogStash::Json::ParserError: Invalid UTF-8 start byte 0x9f
 at [Source: (byte[])"查询用户详情接口"; line: 1, column: 3]>}
[2021-10-15T15:49:28,946][ERROR][logstash.outputs.elasticsearch][main][862f1cf74fcefb4312e0b0aa2e9fdf074e2e77c675e5c24e9b7a04d1054f1947] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"%{aName}-log_2021_10", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x6527baeb>], :response=>{"index"=>{"_index"=>"%{aName}-log_2021_10", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"invalid_index_name_exception", "reason"=>"Invalid index name [%{aName}-log_2021_10], must be lowercase", "index_uuid"=>"_na_", "index"=>"%{aName}-log_2021_10"}}}}
[2021-10-15T15:49:29,446][WARN ][logstash.filters.json    ][main][52e7421840cd9e07ac4081f9b005972562be1954f89c4f4bc5e30c5c19ebae78] Error parsing json {:source=>"message", :raw=>"spAuthAttributes ==========> {\"chainCodes\":\"60d9acb97314488689b899f4495e0857,3d01cb64635b46d4955606e6b356af70,c3283b04c2ad48a09d770b5a96b96a4c,850ff333c6664138b38344a75545ce8e\",\"customLoginStyle\":\"default\",\"samlEntityID\":\"portal\",\"securityLevel\":\"1\",\"spCode\":\"portal\",\"spLocalLogoutUrl\":\"\",\"spName\":\"portal\",\"spOsType\":0,\"tokenValidPeriod\":36000,\"url\":\"https://i.gt.cn\",\"userAttr\":\"\"}", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'spAuthAttributes': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"spAuthAttributes ==========> {"chainCodes":"60d9acb97314488689b899f4495e0857,3d01cb64635b46d4955606e6b356af70,c3283b04c2ad48a09d770b5a96b96a4c,850ff333c6664138b38344a75545ce8e","customLoginStyle":"default","samlEntityID":"portal","securityLevel":"1","spCode":"portal","spLocalLogoutUrl":"","spName":"portal","spOsType":0,"tokenValidPeriod":36000,"url":"https://i.gt.cn","userAttr":""}"; line: 1, column: 18]>}
[2021-10-15T15:49:29,447][WARN ][logstash.filters.json    ][main][52e7421840cd9e07ac4081f9b005972562be1954f89c4f4bc5e30c5c19ebae78] Error parsing json {:source=>"message", :raw=>"redis执行时间：{}1", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'redis执行时间': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"redis执行时间：{}1"; line: 1, column: 21]>}

2、处理方法
这是在处理过程中有日志没有按照要进行打印，我们需要新开一个logstash把异常的日志打印出来；
配置如下：

[root@k8s-master1 conf.d]# more aName1.conf 
input{
    kafka{
           bootstrap_servers => "10.152.17.50:9092"
           group_id => "gt-scs-log-new"
           topics => "gt-scs-log"
           consumer_threads => 1
           decorate_events => true
           auto_offset_reset => "latest"
           type => "gt-scs"
    }
}
filter {
    if [type] == "gt-scs" {
      mutate { 
                add_field => { "types" => "%{type}"}
             }
      json {
                source => "message"
          }
      json {
                source => "message"
          }

    }
}
output { 
  if [tags] { 
        stdout {
                codec => "rubydebug"
        }
  }
}

3、结果

标签：index,gt,10,parsing,json,2021,Error,logstash
来源： https://blog.csdn.net/qq_31555951/article/details/120784966

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。