“Flink实时数据处理”的版本间的差异
来自CloudWiki
Xiaolongyang(讨论 | 贡献) (→1.pom.xml配置,打开IDEA) |
Xiaolongyang(讨论 | 贡献) (→1.pom.xml配置,打开IDEA) |
||
第28行: | 第28行: | ||
(2)右下角都会出现 Maven projects need to be imported(项目需要导入依赖)<IDEA2020版本取消了导入默认依赖提示> | (2)右下角都会出现 Maven projects need to be imported(项目需要导入依赖)<IDEA2020版本取消了导入默认依赖提示> | ||
− | (3)修改pom. | + | (3)修改pom.xml默认依赖(根据以下代码进行局部更改) |
<properties> | <properties> | ||
第48行: | 第48行: | ||
</plugins> | </plugins> | ||
</build> | </build> | ||
+ | |||
+ | (4)向pom.xml中的<dependencies></dependencies>添加依赖 | ||
+ | |||
+ | <!--flink-steaming依赖:流计算--> | ||
+ | <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-scala --> | ||
+ | <dependency> | ||
+ | <groupId>org.apache.flink</groupId> | ||
+ | <artifactId>flink-streaming-scala_2.11</artifactId> | ||
+ | <version>1.10.1</version> | ||
+ | </dependency> | ||
+ | <!--flink-client客户端依赖--> | ||
+ | <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients --> | ||
+ | <dependency> | ||
+ | <groupId>org.apache.flink</groupId> | ||
+ | <artifactId>flink-clients_2.11</artifactId> | ||
+ | <version>1.10.1</version> | ||
+ | </dependency> | ||
+ | |||
+ | <!--flink连接kafka依赖--> | ||
+ | <dependency> | ||
+ | <groupId>org.apache.flink</groupId> | ||
+ | <artifactId>flink-connector-kafka_2.11</artifactId> | ||
+ | <version>1.10.1</version> | ||
+ | </dependency> | ||
+ | |||
+ | <!--mysql依赖:将数据存入mysql中--> | ||
+ | <dependency> | ||
+ | <groupId>mysql</groupId> | ||
+ | <artifactId>mysql-connector-java</artifactId> | ||
+ | <version>5.1.45</version> | ||
+ | </dependency> | ||
+ | |||
+ | <!--添加定义日志格式--> | ||
+ | <dependency> | ||
+ | <groupId>org.slf4j</groupId> | ||
+ | <artifactId>slf4j-log4j12</artifactId> | ||
+ | <version>1.7.7</version> | ||
+ | <scope>runtime</scope> | ||
+ | </dependency> | ||
+ | <dependency> | ||
+ | <groupId>log4j</groupId> | ||
+ | <artifactId>log4j</artifactId> | ||
+ | <version>1.2.17</version> | ||
+ | <scope>runtime</scope> | ||
+ | </dependency> |
2020年8月2日 (日) 10:34的版本
Flink实时统计目标:
1.编程读取kafka缓存数据
2.商品实时统计:
(1)每日销售额,每日访问流量实时统计
(2)每日每家门店总销售额实时统计
(3)每日商品销量实时统计,并选出10大热销商品
(实现过程中,时间单位设置为1~2分钟)
配置开发环境:intellij & IDEA && Maven &&JDK
1.pom.xml配置,打开IDEA<我这里使用的是2018.3.6版本>
(1)首先创建一个新项目Create New Project,操作如下
(2)右下角都会出现 Maven projects need to be imported(项目需要导入依赖)<IDEA2020版本取消了导入默认依赖提示>
(3)修改pom.xml默认依赖(根据以下代码进行局部更改)
<properties> <maven.compiler.target>1.8</maven.compiler.target> <maven.compiler.source>1.8</maven.compiler.source> </properties>
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.6.1</version> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> </plugins> </build>
(4)向pom.xml中的<dependencies></dependencies>添加依赖
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-scala_2.11</artifactId> <version>1.10.1</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.10.1</version> </dependency>
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-connector-kafka_2.11</artifactId> <version>1.10.1</version> </dependency>
<dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.45</version> </dependency>
<dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>1.7.7</version> <scope>runtime</scope> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> <scope>runtime</scope> </dependency>