ELK 配置地图显示IP来源

2018/07/06 Linux

logstash 配置 nginx access 访问日志地图;

ELk 配置地图显示IP来源

logstash 配置 nginx access 访问日志地图;

1.1 安装 logstash 的 geoip 插件

$ /usr/local/logstash/bin/logstash-plugin install logstash-filter-geoip
$ yum -y install GeoIP-data

1.2 编辑kibana配置文件

编辑kibana配置文件kibana.yml,最后面添加

$ vim /etc/kibana/kibana.yml

tilemap.url: 'http://webrd02.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=7&x={x}&y={y}&z={z}'

1.3 在logstash服务器下载IP地址归类查询库

$ cd /usr/local/logstash/config/
$ wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz
$ gunzip GeoLite2-City.mmdb.gz

1.4 编辑 logstash 配置文件

filter {
    grok {
        match => { "message" => "%{NGINXACCESS}" }
    }

    geoip {
        source => "clientip"
        target => "geoip"
        database => "/usr/local/logstash/config/GeoLite2-City.mmdb"
        add_field => ["[geoip][coordinates]","%{[geoip][longitude]}"]
        add_field => ["[geoip][coordinates]","%{[geoip][latitude]}"]
     }

    mutate {
        convert => [ "[geoip][coordinates]", "float"]
    } 
}

配置解释

  • geoip: IP查询插件;

  • source: 需要通过geoip插件处理的field,一般为ip,生产环境中如果查询nginx访问用户,需先将客户端ip过滤出来,然后这里填clientip即可;

  • target: 解析后的Geoip地址数据,应该存放在哪一个字段中,默认是geoip这个字段;

  • database: 指定下载的数据库文件;

  • add_field: 这里两行是添加经纬度,地图中地区显示是根据经纬度来识别;

1.5 配置 logstash 输出到 elasticsearch

output {
    if [type] == "yj-nginx-access"{
        elasticsearch {
           action => "index"
           hosts => ["172.18.100.71:9200","172.18.100.72:9200","172.18.100.73:9200"]
           index => "logstash-yj-nginx-access-%{+YYYY.MM.dd}"
        }
    }

    if [type] == "fp-nginx-access"{
        elasticsearch {
           action => "index"
           hosts => ["172.18.100.71:9200","172.18.100.72:9200","172.18.100.73:9200"]
           index => "logstash-fp-nginx-access-%{+YYYY.MM.dd}"
        }
    }

    if [type] == "yj-nginx-error"{
        elasticsearch {
           action => "index"
           hosts => ["172.18.100.71:9200","172.18.100.72:9200","172.18.100.73:9200"]
           index => "logstash-yj-nginx-error-%{+YYYY.MM.dd}"
        }
    }

    if [type] == "fp-nginx-error"{
        elasticsearch {
           action => "index"
           hosts => ["172.18.100.71:9200","172.18.100.72:9200","172.18.100.73:9200"]
           index => "logstash-fp-nginx-error-%{+YYYY.MM.dd}"
        }
    }
}

这里需要注意的是 output 输出到 elasticsearch 中 index 要以 logstash-开头否侧会出现以下错误,目前没找到解决办法。

1.6 配置 kibana 图形

Search

    Table of Contents