基于elsearch比价引擎

基于elsearch比价引擎

csv文件上传到Elsearch
1.安装Logstash
解压软件包：tar -zxvf logstash-7.3.2.tar.gz
将安装包移动到 mv logstash-7.3.2 /usr/local/
cd /usr/local/
重命名: mv logstash-7.3.2 logstash
进入配置文件目录：/usr/local/logstash/config
原始文件格式：sj.txt

1,1,Apparel_Textiles & Accessories,Apparel,Apparel Design Services,//s.alicdn.com/@sc01/kf/H50cfdfc8a6c4445ab6f3a926a2542c00t.jpg_300x300.jpg,embroidery 
digitizing service 24h on line,1,$1.00-$2.00,Unit,Shen Zhen Happitoo Textile Co._ Ltd.,//happitoo.en.alibaba.com/company_profile.html#top-nav-bar,China,1,Apparel Design Apparel_Textiles Accessories Services

在/usr/local/logstash/config目录下创建abc-es.conf

input {
 file {
   path => ["/root/sj.txt"]
   start_position => "beginning"
 }
}
filter {
 csv {
   separator => ","
   columns => 
["productId","productPage","categoryOne","categoryTwo","categoryThr","productImg","productName","productPrice","productPrices","productUnit","companyName","companyUrl","address","year","tags"]
 } 
}
output {
 elasticsearch {
       hosts => ["master:9200"]
       index => "flight"
 }
}

执行命令：./bin/logstash -f config/abc-es.conf --path.data=/root/abc

将mysql数据上传到Elsearch

首先在数据库中创建一个表:

create table data(
   productId int,
   productPage int,
   categoryOne varchar(300),
   categoryTwo varchar(300),
   categoryThr varchar(300),
   productImg varchar(300),	
   productName varchar(300),
   productPrice float,
   productPrices varchar(300),
   productUnit varchar(300),
   companyName varchar(300),
   companyUrl varchar(300),
   address varchar(300),
   year int,
   tags varchar(300) );

接下来将数据导入到表: load data local infile "/root/sj.txt" into table data FIELDS TERMINATED BY "," LINES TERMINATED BY "\n"

查询发现数据正常：