前後端分離的爬蟲小項目

前後端分離的爬蟲小項目-----EncycProject

---

EncycProject項目使用現在流行的前後端分離技術；此項目創作目的是實踐。利用此項目快速搭建環境和快速入門。該項目的主線是建立寵物社區網站。目前該項目完成第一個模塊：寵物百科；剩餘模塊陸續開發中。（此項目可以變更，第一模塊耦合度低）

---

項目地址

項目GitHub地址

1 項目效果

1.1 效果圖：

爬蟲數據

數據庫數據

前端

待續

2 項目結構圖

3.1 框架整合

3.1.1 pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.5.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.threadnew</groupId>
    <artifactId>encycproject</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>encycproject</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>2.0.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <!--導入webmagic爬蟲的包-->
        <dependency>
            <groupId>us.codecraft</groupId>
            <artifactId>webmagic-core</artifactId>
            <version>0.7.3</version>
        </dependency>
        <dependency>
            <groupId>us.codecraft</groupId>
            <artifactId>webmagic-extension</artifactId>
            <version>0.7.3</version>
        </dependency>
        <!--配置druid-->
        <!-- https://mvnrepository.com/artifact/com.alibaba/druid-spring-boot-starter -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
            <version>1.1.17</version>
        </dependency>
        <!--import solr-->
        <!-- Compile -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-solr</artifactId>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

3.1.2 applicaton.yml

spring:
  data:
    solr:
      host: http://localhost:8080/solr
  datasource:
    name: encycproject
    type: com.alibaba.druid.pool.DruidDataSource
    #druid相關配置
    druid:
      #監控統計攔截的filters
      filters: stat
      driver-class-name: com.mysql.cj.jdbc.Driver
      #基本屬性
      url: jdbc:mysql://127.0.0.1:3306/animal?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=GMT%2B8
      username: root
      password: 1234
      #配置初始化大小/最小/最大
      initial-size: 1
      min-idle: 1
      max-active: 20
      #獲取連接等待超時時間
      max-wait: 60000
      #間隔多久進行一次檢測，檢測需要關閉的空閒連接
      time-between-eviction-runs-millis: 60000
      #一個連接在池中最小生存的時間
      min-evictable-idle-time-millis: 300000
      validation-query: SELECT 'x'
      test-while-idle: true
      test-on-borrow: false
      test-on-return: false
      #打開PSCache，並指定每個連接上PSCache的大小。oracle設爲true，mysql設爲false。分庫分表較多推薦設置爲false
      pool-prepared-statements: false
      max-pool-prepared-statement-per-connection-size: 20
server:
  port: 8100

3.1.3 前端的axios的代理

在config文件夾下的index.js中的proxyTable下添加如下代碼

 '/apis':{
        target: 'http://192.168.43.103:8100/',  // 後臺api
        changeOrigin: true,  //是否跨域
        // secure: true,
        pathRewrite: {
          '^/apis': ''   //需要rewrite的,
        }
      }

 > 未完待續