2023-05 组内技术分享

回归剖析

介绍

回归剖析是一种统计剖析办法，用于树立变量之间的数学联系并对它们进行猜测和操控。回归剖析的首要意图是研讨一个或多个自变量与因变量之间的联系。简略线性回归剖析触及一个自变量和一个因变量，而多元回归剖析触及两个或更多个自变量和一个因变量。

我的了解：回归的意图是进行猜测，比方 2000 – 2020 年鱼塘的产值数据都存在，现在需求对 2020-2030 年鱼塘的产值进行猜测，就能够用回归剖析的算法

线性回归公式：y = ax + b
多元回归公式（举个例子）：y = ax^n + bx^(n-1) + cx^(n-2) + ... + z

• 自变量（年份）：x
• 因变量（产值）：y
• 算法模型训练出来的常量：a、b、c、d ……

在回归剖析中，自变量通常是已知的，并用于猜测或操控因变量。当猜测因变量时，回归剖析可认为未来事件供给牢靠的猜测成果。当操控因变量时，回归剖析能够评价自变量对因变量的效果，以便在未来研讨或试验中应用这些发现。

回归剖析广泛应用于许多范畴，包含经济学、社会科学、医学、生物学、物理学等。一些常见的应用包含市场研讨、风险评价、物理建模、药物研发和轿车工程。

具体完成

1、导入依靠

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-math3</artifactId>
    <version>3.6.1</version>
</dependency>

2、导入东西类

/**
 * 用于核算回归曲线
 * 第一步：核算拟合数据 calcFittingData(x,y)
 * 第二步：依据拟合数据，核算对应 x1 的回归数据 y1
 */
public class Regression {
    /**
     * 核算拟合数据，用于多项式的核算
     */
    public static double[] calcFittingData(List<Double> x, List<Double> y){
        // 构建数据点集合
        WeightedObservedPoints points = new WeightedObservedPoints();
        for (int i = 0; i < x.size(); i++) {
            points.add(x.get(i), y.get(i));
        }
        // 选择多项式次数
        int degree = 3;
        // 构建多项式回归器
        PolynomialCurveFitter fitter = PolynomialCurveFitter.create(degree);
        // 拟合数据
        return fitter.fit(points.toList());
    }
    /**
     * 依据拟合数据，核算对应 x1 的回归数据 y1
     * @param fittingData 上一步得到的拟合数据
     * @param x 需求猜测的点位
     * @return 猜测的值
     */
    public static double calcRegressionValue(double[] fittingData, double x){
        double result = 0.0;
        for (int i = fittingData.length - 1; i >= 0; i--) {
            result = result * x + fittingData[i];
        }
        return NumberUtil.round(result, 4).doubleValue();
    }
}

3、生成历史数据

public class DataGenerate {
    public static void main(String[] args) {
//        List<Double> x = DoubleStream.iterate(2000, n -> n + 0.2).limit(100).boxed().collect(Collectors.toList());
//        List<Double> y = DoubleStream.iterate(1, n-> n + RandomUtil.randomDouble(-10, 10)).limit(100).boxed().collect(Collectors.toList());
        List<Double> x = JSON.parseArray("[2000.0, 2000.2, 2000.4, 2000.6000000000001, 2000.8000000000002, 2001.0000000000002, 2001.2000000000003, 2001.4000000000003, 2001.6000000000004, 2001.8000000000004, 2002.0000000000005, 2002.2000000000005, 2002.4000000000005, 2002.6000000000006, 2002.8000000000006, 2003.0000000000007, 2003.2000000000007, 2003.4000000000008, 2003.6000000000008, 2003.8000000000009, 2004.000000000001, 2004.200000000001, 2004.400000000001, 2004.600000000001, 2004.800000000001, 2005.0000000000011, 2005.2000000000012, 2005.4000000000012, 2005.6000000000013, 2005.8000000000013, 2006.0000000000014, 2006.2000000000014, 2006.4000000000015, 2006.6000000000015, 2006.8000000000015, 2007.0000000000016, 2007.2000000000016, 2007.4000000000017, 2007.6000000000017, 2007.8000000000018, 2008.0000000000018, 2008.2000000000019, 2008.400000000002, 2008.600000000002, 2008.800000000002, 2009.000000000002, 2009.200000000002, 2009.4000000000021, 2009.6000000000022, 2009.8000000000022, 2010.0000000000023, 2010.2000000000023, 2010.4000000000024, 2010.6000000000024, 2010.8000000000025, 2011.0000000000025, 2011.2000000000025, 2011.4000000000026, 2011.6000000000026, 2011.8000000000027, 2012.0000000000027, 2012.2000000000028, 2012.4000000000028, 2012.6000000000029, 2012.800000000003, 2013.000000000003, 2013.200000000003, 2013.400000000003, 2013.600000000003, 2013.8000000000031, 2014.0000000000032, 2014.2000000000032, 2014.4000000000033, 2014.6000000000033, 2014.8000000000034, 2015.0000000000034, 2015.2000000000035, 2015.4000000000035, 2015.6000000000035, 2015.8000000000036, 2016.0000000000036, 2016.2000000000037, 2016.4000000000037, 2016.6000000000038, 2016.8000000000038, 2017.0000000000039, 2017.200000000004, 2017.400000000004, 2017.600000000004, 2017.800000000004, 2018.000000000004, 2018.2000000000041, 2018.4000000000042, 2018.6000000000042, 2018.8000000000043, 2019.0000000000043, 2019.2000000000044, 2019.4000000000044, 2019.6000000000045, 2019.8000000000045]", Double.class);
        List<Double> y = JSONArray.parseArray("[1.0, 10.42906325168871, 11.26071129761812, 14.634243269661564, 17.307783514340677, 14.445901316527763, 13.293616154532192, 23.237659619927054, 14.457660333900618, 5.921597276626496, 7.702984596970076, 6.524844239013698, -1.5960356170082886, -5.560994664449881, -6.2830433095772715, -13.337225004507358, -13.216263420456446, -17.304496396195166, -23.892387031375236, -14.371445291535515, -4.567274662509281, -13.70839788551179, -7.043528004304555, -2.5458381300587405, 7.162985271186795, 0.9621560164152587, 8.656852150628934, 8.001760651158149, 4.665505015132808, 11.06667379861096, 15.935532921303357, 11.772343753661755, 17.240023933014392, 20.725849565650947, 26.530486782767607, 33.65381775688789, 30.860190145623353, 33.64173445019578, 36.68001666381011, 43.041099209116034, 49.34199357956133, 49.28431164607146, 47.42987740061483, 46.181005186790195, 50.916612065393295, 55.92817502056798, 58.68406683831894, 57.056091061056144, 51.2465621566577, 59.54939721768027, 57.61449805533475, 49.33510392003578, 50.268702082462724, 53.05629699476951, 57.38147080259562, 55.54787439905726, 65.27895027772341, 58.92933730696279, 52.56821849857317, 55.73207898990508, 63.79709000231691, 56.27245713523001, 50.62707899653262, 59.19803055821095, 53.067329345898614, 55.80932675121849, 54.98240075838871, 46.25440822982649, 49.97837740104257, 59.55490162760393, 51.62529700925586, 45.82178379468647, 39.86561267351533, 46.79088499145376, 38.85287321384413, 40.855002655929795, 31.77199874359535, 34.84457819539462, 28.1545543743269, 28.115016953106398, 36.65232853824082, 45.942482890756054, 53.09468749771434, 60.87143535541702, 55.86161670529165, 59.13630259872876, 56.51387974596331, 55.00271108850981, 60.08807639190195, 54.34522427227326, 58.96382641685028, 67.15709893357109, 75.5639618703819, 65.81677636550283, 58.92258534065034, 49.22604248835671, 44.813368860886236, 36.6180611087144, 46.29321257791983, 56.056426120193535]", Double.class);
        double[] fittingData = Regression.calcFittingData(x, y);
        List<Double> y1 = x.stream().map(e -> {
            return Regression.calcRegressionValue(fittingData, e);
        }).collect(Collectors.toList());
        System.out.println("x = " + x);
        System.out.println("y = " + y);
        System.out.println("y1 = " + y1);
    }
}

4、制作图形

import matplotlib.pyplot as plt
x = [2000.0, 2000.2, 2000.4, 2000.6000000000001, 2000.8000000000002, 2001.0000000000002, 2001.2000000000003, 2001.4000000000003, 2001.6000000000004, 2001.8000000000004, 2002.0000000000005, 2002.2000000000005, 2002.4000000000005, 2002.6000000000006, 2002.8000000000006, 2003.0000000000007, 2003.2000000000007, 2003.4000000000008, 2003.6000000000008, 2003.8000000000009, 2004.000000000001, 2004.200000000001, 2004.400000000001, 2004.600000000001, 2004.800000000001, 2005.0000000000011, 2005.2000000000012, 2005.4000000000012, 2005.6000000000013, 2005.8000000000013, 2006.0000000000014, 2006.2000000000014, 2006.4000000000015, 2006.6000000000015, 2006.8000000000015, 2007.0000000000016, 2007.2000000000016, 2007.4000000000017, 2007.6000000000017, 2007.8000000000018, 2008.0000000000018, 2008.2000000000019, 2008.400000000002, 2008.600000000002, 2008.800000000002, 2009.000000000002, 2009.200000000002, 2009.4000000000021, 2009.6000000000022, 2009.8000000000022, 2010.0000000000023, 2010.2000000000023, 2010.4000000000024, 2010.6000000000024, 2010.8000000000025, 2011.0000000000025, 2011.2000000000025, 2011.4000000000026, 2011.6000000000026, 2011.8000000000027, 2012.0000000000027, 2012.2000000000028, 2012.4000000000028, 2012.6000000000029, 2012.800000000003, 2013.000000000003, 2013.200000000003, 2013.400000000003, 2013.600000000003, 2013.8000000000031, 2014.0000000000032, 2014.2000000000032, 2014.4000000000033, 2014.6000000000033, 2014.8000000000034, 2015.0000000000034, 2015.2000000000035, 2015.4000000000035, 2015.6000000000035, 2015.8000000000036, 2016.0000000000036, 2016.2000000000037, 2016.4000000000037, 2016.6000000000038, 2016.8000000000038, 2017.0000000000039, 2017.200000000004, 2017.400000000004, 2017.600000000004, 2017.800000000004, 2018.000000000004, 2018.2000000000041, 2018.4000000000042, 2018.6000000000042, 2018.8000000000043, 2019.0000000000043, 2019.2000000000044, 2019.4000000000044, 2019.6000000000045, 2019.8000000000045]
y = [1.0, 10.42906325168871, 11.26071129761812, 14.634243269661564, 17.307783514340677, 14.445901316527763, 13.293616154532192, 23.237659619927054, 14.457660333900618, 5.921597276626496, 7.702984596970076, 6.524844239013698, -1.5960356170082886, -5.560994664449881, -6.2830433095772715, -13.337225004507358, -13.216263420456446, -17.304496396195166, -23.892387031375236, -14.371445291535515, -4.567274662509281, -13.70839788551179, -7.043528004304555, -2.5458381300587405, 7.162985271186795, 0.9621560164152587, 8.656852150628934, 8.001760651158149, 4.665505015132808, 11.06667379861096, 15.935532921303357, 11.772343753661755, 17.240023933014392, 20.725849565650947, 26.530486782767607, 33.65381775688789, 30.860190145623353, 33.64173445019578, 36.68001666381011, 43.041099209116034, 49.34199357956133, 49.28431164607146, 47.42987740061483, 46.181005186790195, 50.916612065393295, 55.92817502056798, 58.68406683831894, 57.056091061056144, 51.2465621566577, 59.54939721768027, 57.61449805533475, 49.33510392003578, 50.268702082462724, 53.05629699476951, 57.38147080259562, 55.54787439905726, 65.27895027772341, 58.92933730696279, 52.56821849857317, 55.73207898990508, 63.79709000231691, 56.27245713523001, 50.62707899653262, 59.19803055821095, 53.067329345898614, 55.80932675121849, 54.98240075838871, 46.25440822982649, 49.97837740104257, 59.55490162760393, 51.62529700925586, 45.82178379468647, 39.86561267351533, 46.79088499145376, 38.85287321384413, 40.855002655929795, 31.77199874359535, 34.84457819539462, 28.1545543743269, 28.115016953106398, 36.65232853824082, 45.942482890756054, 53.09468749771434, 60.87143535541702, 55.86161670529165, 59.13630259872876, 56.51387974596331, 55.00271108850981, 60.08807639190195, 54.34522427227326, 58.96382641685028, 67.15709893357109, 75.5639618703819, 65.81677636550283, 58.92258534065034, 49.22604248835671, 44.813368860886236, 36.6180611087144, 46.29321257791983, 56.056426120193535]
y1 = [0.8864, 0.8444, 0.8631, 0.9407, 1.0758, 1.2667, 1.5119, 1.8097, 2.1586, 2.557, 3.0032, 3.4957, 4.033, 4.6133, 5.2352, 5.897, 6.5971, 7.334, 8.106, 8.9116, 9.7491, 10.6171, 11.5138, 12.4377, 13.3872, 14.3607, 15.3567, 16.3734, 17.4095, 18.4631, 19.5328, 20.617, 21.7141, 22.8224, 23.9404, 25.0665, 26.1992, 27.3367, 28.4776, 29.6202, 30.7629, 31.9042, 33.0424, 34.1761, 35.3034, 36.423, 37.5332, 38.6324, 39.7189, 40.7913, 41.848, 42.8872, 43.9075, 44.9073, 45.8849, 46.8388, 47.7674, 48.669, 49.5422, 50.3853, 51.1966, 51.9747, 52.7179, 53.4247, 54.0934, 54.7225, 55.3103, 55.8553, 56.3559, 56.8105, 57.2175, 57.5753, 57.8823, 58.1369, 58.3376, 58.4827, 58.5707, 58.5999, 58.5688, 58.4758, 58.3192, 58.0976, 57.8092, 57.4526, 57.0261, 56.5281, 55.957, 55.3113, 54.5894, 53.7896, 52.9104, 51.9501, 50.9073, 49.7802, 48.5674, 47.2672, 45.878, 44.3982, 42.8263, 41.1606]
plt.plot(x, y, x, y1)
plt.show()

成果

mybatis-enhance-actable

开源库房：gitee.com/sunchenbin/…

介绍

mybatis-enhance-actable 是一款 mybatis plus 的增强东西，能够依据实体类主动建表，实体类产生修正，会主动同步到数据库，这样修正字段就不用在数据库和实体类中频频切换了，能够较大进步开发功率。

运用这个东西之后，开发形式就能够从面向数据库转变到面向实体类开发了

运用步骤

条件：装备好数据库

<dependency>
  <groupId>mysql</groupId>
  <artifactId>mysql-connector-java</artifactId>
</dependency>

spring.datasource.url=jdbc:mysql://127.0.0.1:3306/config_center?useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true&serverTimezone=Asia/Shanghai
spring.datasource.username=root
spring.datasource.password=11111111
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

---

1、导入依靠

<dependency>
    <groupId>com.gitee.sunchenbin.mybatis.actable</groupId>
    <artifactId>mybatis-enhance-actable</artifactId>
    <version>1.5.0.RELEASE</version>
</dependency>
<dependency>
    <groupId>com.baomidou</groupId>
    <artifactId>mybatis-plus-boot-starter</artifactId>
    <version>3.4.1</version>
</dependency>

2、启动类上参加注解

@MapperScan("com.gitee.sunchenbin.mybatis.actable.dao.*")
@ComponentScan(basePackages = {"com.gitee.sunchenbin.mybatis.actable.manager.*"})
@SpringBootApplication
public class MySpringBootApplication {
    public static void main(String[] args) {
        SpringApplication.run(MySpringBootApplication.class, args);
    }
}

3、增加装备

actable.table.auto=update
actable.model.pack=com.rabbit.entity
actable.database.type=mysql
mybatis-plus.mapper-locations=classpath*:com/gitee/sunchenbin/mybatis/actable/mapping/*/*Mapper.xml

4、实体类运用注解

---

缺点

• 装备太多，装备不行简略，没有用到 springboot 主动装配特性
• 强依靠于 mybatis-plus，不行解耦，类似这种底层结构，依靠越少越好，其实完全能够运用 jdbc 去做
• 主动建表、改表、删表功能不安全，可能我对这个项目部署，不知道他的开发流程，我先在数据库新建好了表，项目一启动因为没有增加实体类把我的表全给删除了。。。。。。
• 许多公司不答应项目直接操作数据库，供给的数据库只要查询权限，如果需求修正表，需求开发提 db 工单，经过层层审核，最后由 dba 进行履行，在这一层，许多公司都不会运用这

完成原理

1、获取数据表的元数据

数据库每张表的元数据都在 information_schema 这个 mysql 自带的库里存在，能够直接取出来

select * from information_schema.columns where table_name = 'pilot_task' and table_schema = (select database())

2、扫描代码中的实体类，运用反射提取字段、注解信息

3、将上面两堆数据进行比对，找出需求新建的表、需求修正、需求删除的表

4、拼接 sql 语句

最后

底座也有主动建表、改表功能，可是我发现有个问题，表注释不会生成

MapStruct

github:github.com/mapstruct/m…

介绍

389c3b3079459d553a01061fb41eab17.pdf

在开发过程中，我们经常会进行 AO、VO、DTO、PO 之间的特点复制，常用的办法有以下几种：

• spring 的BeanUtils.copyProperties()
• hutool 的 Convert、BeanUtil 东西类
• 手动的 set 值

毫无疑问，手动的 set 值速度是最快的，可是代码就比较多；其他两种底层都是用了反射，在速度上略逊一筹，并且只能复制相同字段称号的特点，运用比较有局限性

下面介绍一下 MapStruct 的长处：

• 而 MapStruct 他的性能是远高于 BeanUtils 的，下面是测试成果：

• 并且 MapStruct 还满足灵活，能够应对特点复制的各种场景
• 运用满足简略

运用步骤

1、导入依靠

<dependency>
    <groupId>org.mapstruct</groupId>
    <artifactId>mapstruct</artifactId>
    <version>1.5.3.Final</version>
</dependency>
<build>
  <plugins>
      <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.8.1</version>
          <configuration>
              <source>8</source>
              <target>8</target>
              <annotationProcessorPaths>
                  <path>
                      <groupId>org.mapstruct</groupId>
                      <artifactId>mapstruct-processor</artifactId>
                      <version>1.5.3.Final</version>
                  </path>
                  <path>
                      <groupId>org.projectlombok</groupId>
                      <artifactId>lombok</artifactId>
                      <version>1.18.26</version>
                  </path>
                  <path>
                      <groupId>org.projectlombok</groupId>
                      <artifactId>lombok-mapstruct-binding</artifactId>
                      <version>0.2.0</version>
                  </path>
              </annotationProcessorPaths>
          </configuration>
      </plugin>
  </plugins>
</build>

2、编写两个完成类

@Data
public class DemoPO {
    private String userName;
}
@Data
public class DemoVO {
    private String userName;
}

3、编写转化器

@Mapper
public interface DemoConvert {
    DemoConvert INSTANCE = Mappers.getMapper(DemoConvert.class);
    DemoVO toVo(DemoPO po);
}

4、main 办法运行

public static void main(String[] args) {
    DemoPO demoPO = new DemoPO();
    demoPO.setUserName("小明");
    DemoVO vo = DemoConvert.INSTANCE.toVo(demoPO);
    System.out.println(vo);
}

杂乱场景映射

1、场景一：PO 和 VO 字段称号不一致

@Data
public class DemoPO {
    private String userName;
}
@Data
public class DemoVO {
    private String name;
}
@Mapper
public interface DemoConvert {
    DemoConvert INSTANCE = Mappers.getMapper(DemoConvert.class);
    @Mapping(target = "name", source = "userName")
    DemoVO toVo(DemoPO po);
}

2、多层嵌套映射

@Data
public class DemoPO {
    private String userName;
}
@Data
public class DemoVO {
    private String name;
}
@Mapper
public interface DemoConvert {
    DemoConvert INSTANCE = Mappers.getMapper(DemoConvert.class);
    @Mapping(target = "name", source = "userName")
    DemoVO toVo(DemoPO po);
    List<DemoVO> toVoList(List<DemoPO> poList);
}
public class DemoMain {
    public static void main(String[] args) {
        DemoPO demoPO = new DemoPO();
        demoPO.setUserName("小明");
        List<DemoVO> voList = DemoConvert.INSTANCE.toVoList(Collections.singletonList(demoPO));
        System.out.println(voList);
    }
}

如果嵌套有三层、四层、五层，同样的原理

3、日期时间格式化

@Data
public class DemoPO {
    private Date createDate;
}
@Data
public class DemoVO {
    private String date;
}
@Mapper
public interface DemoConvert {
    DemoConvert INSTANCE = Mappers.getMapper(DemoConvert.class);
    @Mapping(target = "date", source = "createDate", dateFormat = "yyyy-MM-dd")
    DemoVO toVo(DemoPO po);
}

4、数字格式化

@Data
public class DemoPO {
    private Double num;
}
@Data
public class DemoVO {
    private String num;
}
@Mapper
public interface DemoConvert {
    DemoConvert INSTANCE = Mappers.getMapper(DemoConvert.class);
    @Mapping(target = "num", source = "num", numberFormat = "#0.00")
    DemoVO toVo(DemoPO po);
}

5、默认值

@Data
public class DemoPO {
    private String def;
}
@Data
public class DemoVO {
    private String def;
}
@Mapper
public interface DemoConvert {
    DemoConvert INSTANCE = Mappers.getMapper(DemoConvert.class);
    @Mapping(target = "def", defaultValue = "hello world")
    DemoVO toVo(DemoPO po);
}

---

---

6、自定义转化

把目标转成 json 字符串

@Data
public class DemoPO {
    private Map<String, String> map;
}
@Data
public class DemoVO {
    private String map;
}
public class MapConvertUtil {
    @Named("toJson")
    public String toJson(Map<String, String> map){
        return JSONUtil.toJsonStr(map);
    }
}
@Mapper(uses = MapConvertUtil.class)
public interface DemoConvert {
    DemoConvert INSTANCE = Mappers.getMapper(DemoConvert.class);
    @Mapping(target = "map", source = "map", qualifiedByName="toJson")
    DemoVO toVo(DemoPO po);
}
public class DemoMain {
    public static void main(String[] args) {
        HashMap<String, String> map = new HashMap<>();
        map.put("aaa", "bbb");
        DemoPO demoPO = new DemoPO();
        demoPO.setMap(map);
        DemoVO vo = DemoConvert.INSTANCE.toVo(demoPO);
        System.out.println(vo);
    }
}

原理

MapStruct 的完成原理和 Lombok 类似，都是基于 JSR 269 标准，在源码编译期间生成字节码

JSR 269 它供给了一种机制，答应开发人员编写注解处理器来处理 Java 源代码中的注解，然后生成额定的 Java 代码。

其实便是 java 编译的过程中，会经过 jdk 的 ServiceLoader 这个类，扫描类途径下的META-INF/services/javax.annotation.processing.Processor这个文件，取出里边的类名进行类加载和实例化，然后调用接口的 process 办法去生成对应的字节码

画图解释

java 程序编译运行阅历以下几个流程

源码编译成 class 文件其实是个比较杂乱的过程