14.1 创建 idea 模块

模块名: gmall-dw2es

14.1.1 导入依赖:

<dependencies>
    <dependency>
        <groupId>com.atguigu.dw</groupId>
        <artifactId>gmall-common</artifactId>
        <version>1.0-SNAPSHOT</version>
    </dependency>

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
    </dependency>

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-hive_2.11</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
    </dependency>

    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.27</version>
    </dependency>
</dependencies>

14.1.2 copy hie-site.xml 的内容

在resources目录中添加hive-site.xml从而可以让 SparkSql 来找到我们的仓库.

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://hadoop201:3306/metastore?createDatabaseIfNotExist=true</value>
        <description>JDBC connect string for a JDBC metastore</description>
    </property>

    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>Driver class name for a JDBC metastore</description>
    </property>

    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
        <description>username to use against metastore database</description>
    </property>

    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>aaa</value>
        <description>password to use against metastore database</description>
    </property>
    <!-- zookeeper 地址-->
    <property>
        <name>hive.zookeeper.quorum</name>
        <value>hadoop201,hadoop202,hadoop203</value>
        <description>The list of ZooKeeper servers to talk to. This is only needed for read/write locks.</description>
    </property>
    <!-- zookeeper 端口号 -->
    <property>
        <name>hive.zookeeper.client.port</name>
        <value>2181</value>
        <description>The port of ZooKeeper servers to talk to. This is only needed for read/write locks.</description>
    </property>
</configuration>