Camel Spring Boot Framework(Open Source To The World)

Chinese Version

Camel is a set of customized SpringBoot development framework (Java), to improve the development efficiency and stable program quality, 
playing a pivotal role During the development cycle, is indeed the necessary medicine for home and travel. especially, for small start-up 
companies,The programmer who work here is so lucky to see this framework. Then look at the customer feedback, although there is a certain 
boasting, but it can survival for five years and is chosen, and is widely used in the company, there are worthy of study.
More: After resigning from the company, I re-factory and re-wrote the project. I hope you will like it!
=
The new staff said: "I have worked in so many companies, and your framework is so practical and simple that I can take it!"
The former staff said: "Go out to take this framework, super useful, if you can, the value of 100 million!"
Four YEARS of competitors and close partners Ma said: "Use can not leave, the weight is just right, the company on you (framework) 
the most reliable parter!"
=
Code is still being tested, Please wait patiently....

Preface

Just do it. Maybe the framework will survival a few years！ —- Youcai 2016-06-01
This document is dedicated to you who worked hard in an ordinary position in 2018. —- Youcai 2019-01-26
The Camel framework is at least 5-10 years ahead of the world with zero configuration technology! —- Youcai 2022-02-18
To upgrade to SpingBoot2.5, history hands us the technical optimization task! —- Youcai 2022-04-20
Rewrite and open source Camel framework is the inevitable output of the enterprise to export talents to the society！—- Youcai 2022-08-10

Begining Camel

The Camel framework is built on SpringBoot2.5.9 （https://spring.io/projects/spring-boot/#support ），main concern three things：

Development environment complex configuration and startup problems, such as development environment, test environment, production environment and other complex configuration files and startup, etc;
Multi-data source zero configuration elegant solution, multi-data source configuration is no longer a problem, one of the world's most elegant solutions (leading the world by five to ten years)；
Optimal parameter combination and optimization, which has experienced the baptism of tens of millions or even hundreds of millions of levels of throughput(request), etc.

The fastest way to learn a framework is to run the code, so let's start with it and follow it step by step：

SpringBoot 2.5.9 was first introduced:

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.5.9</version>
</parent>

Introduce the Camel framework again (see the release record of the GIT branch)：

<dependency>
    <groupId>cc.goodrain.framework</groupId>
    <artifactId>camel-spring-boot-starter</artifactId>
    <version>2.5.9.32-RELEASE</version>
</dependency>

At this point, you're ready to use all of Camel's capabilities。

Warm tips (if you are familiar with package management, ignore the following tips)：

This framework has introduced bellowing dependencies, your project no longer need to repeat the introduction, has been repeatedly introduced, suggest to delete, afraid of inconsistent version(Use the IDE to view the final JAR version and see if multiple versions have been introduced at the same time):

commons-lang3
spring-boot-starter-web
spring-boot-starter-undertow
spring-boot-starter-jdbc
spring-boot-starter-data-redis
mysql-connector-java
tomcat-jdbc
HikariCP
jedis
jackson-core
jackson-databind
kafka-clients
elasticsearch-rest-high-level-client
elasticsearch
elasticsearch-rest-client

Springboot predefined dependencies(refer to spring-boot-dependencies-2.5.9.pom)，When referring to these dependencies, it is best to use the default version, as when Springboot's predefined Gson denpendency are introduced, Do this without defining the '' property:
```
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
</dependency>
```
If you want to overwrite the Springboot predefined dependencies, define the corresponding version attributes in the root pom.xml of your project, not in the submodule. If you are using ElasticSearch, versions 6 and 7 are very different, When selecting different elasticSearch versions, define and override the global uniform version number in root pop.xml, otherwise you may get errors. Do it such as：
```
<properties>
    <elasticsearch.version>6.2.2</elasticsearch.version>
</properties>
```

1.1、camel-boot(Complex configuration and Startup problems for development environments -> Solutions)

Camel-boot inherits almost all the advantages of Spring-Boot, while stipulating and stabilizing the parameters of some components and providing optimization schemes. Camel-boot is very suitable for excellent programmers (especially you) who pursue high development efficiency and quality.

Use the CamelSpringApplication boot class instead of the SpringBoot default boot class as shown below：

@SpringBootApplication(scanBasePackages = {"cc.goodrain.demo"})
@ConfigurationProperties(prefix = "cc.goodrain.demo")
@CamelBootModule("example")
@CamelBootWebEnvironment(CamelBootWebEnvironment.CamelBootWebEnvironmentType.Yes) //You don't have to write this, the default is Yes

@EnableSwagger2
public class ApplicationMain {
    public static void main(String[] args) {
        new CamelSpringApplication(ApplicationMain.class).run(args);
    }
}

Running the above 'ApplicationMain' will output the relevant help information (some of which is summarized below)：

欢迎使用Camel框架，启动配置文件如下：
配置文件表达式：application(-部署环境)(-项目模块)?(-机房标识)?.properties
部署环境：值仅能是local|dev|test|stage|prod之一，值从环境变量中DWENV读取，不配置默认是local
项目模块(选配)：使用注解类@CamelBootModule(x)标记@SpringBootApplication的启动类
机房标识(选配)：使用JVM参数来-Dserver.region=x来标记(现在几乎没有人用这个项)
::
>>>当前启动配置文件是：application.properties
>>>当前启动配置文件是：application-local.properties
>>>当前启动配置文件是：application-local-web.properties
>>>程序按顺序处理上面的配置文件，相同的配置项，后面的会覆盖前面的(last-wins strategy)！！！

The following is a supplementary description of the above parameters:

The development environment, modules, and multi-zone deployment parameters are configured as follows；

Development environment parameter Settings：-DDWENV=local|dev|test|stage|prod（JVM startup parameters）
Development module parameter setting：@CamelBootModule("x") （This parameter is optional. It is used for multiple maven modules in the same project）
Set deployment region parameters：-Dserver.region=x （Optional, mainly to facilitate multiple region deployment）

Multilevel configuration file inheritance and overwrite read order and priority(last-wins strategy)；

application.properties 
application-部署环境.properties (If there is)
application-部署环境-项目模块.properties （If there is）

Configure parameters for Web and non-Web environments；

@CamelBootWebEnvironment(CamelBootWebEnvironment.CamelBootWebEnvironmentType.Yes) The WEB server is started
@CamelBootWebEnvironment(CamelBootWebEnvironment.CamelBootWebEnvironmentType.NO) The WEB server is not started

Other parameter conventions；

Actual project usage tests found that Undertow performed better, especially with a smaller memory footprint, and CLOSE_WAIT had fewer TCP problems with high concurrency，
This framework is changed from the default Tomcat server to the Undertow server.

1.2、 camel-data（Multi-data source zero configuration elegant solution）

Before explaining how to use 'Camel-data', first know about the latest version of 'Springboot' how zero configuration.

Take a look at the native step of a Mysql program: First, you configure the DataSource file (account, password, database information, etc.), then you write the DataSourceProperties program, then you write the DataSource program such as DataSource, and then you write JdbcTemplate program, at least 4 steps should be done, see details: https://docs.spring.io/spring-boot/docs/2.5.9/reference/html/howto.html#howto.data-access.configure-two-datasources.

If we take our project as an example, there are about 20 mysql data sources, and the native program will have to write more than 20*4=80 programs, which I believe is a very difficult task for development and maintenance.What Camel-Data does is give you a one-step approach to generating relevant programs (called beans here) that you can use directly, allowing us to focus more on the business and increase productivity.

According to my personal development experience, CAMEL framework integrates the most used Mysql, Redis, Kafka, Elasticsearch four data sources. Other data sources can be configured as official or implemented yourself. Finally, with this framework, you can still use the default data source configuration.

1.2.1、 Mysql usage instructions

Configure the mysql database configuration as follows：

camel.mysql.main.url=127.0.0.1:3306:dbname
camel.mysql.main.username=Youcai
camel.mysql.main.password=Youcai_like_demo
camel.mysql.main.initialSize=30
camel.mysql.main.minIdle=30
camel.mysql.main.maxIdle=50
camel.mysql.main.maxActive=50
camel.mysql.main.maxAge=28800000
camel.mysql.main.testOnBorrow=true
camel.mysql.main.testWhileIdle=true
#The validationQuery in here suggests using this, which works wonders for cloud vendor Failover database switches
camel.mysql.main.validationQuery=select case when @@read_only + @@innodb_read_only = 0 then 1 else (select table_name from information_schema.tables limit 2) end as `1`
camel.mysql.main.validationInterval=30000

Note that the keyword configured above is main, and the framework will use this keyword to generate the following beans, which can be injected immediately, Suggested that injection mainCamelJdbcTemplate or mainNamedParameterJdbcTemplate use。

mainMysqlDataSource, mainJdbcTemplate, mainCamelJdbcTemplate, mainNamedParameterJdbcTemplate

Here is a sample：

@Repository
public class PeopleDaoImpl implements PeopleDao {

@Autowired
@Qualifier("mainCamelJdbcTemplate")
CamelJdbcTemplate mainCamelJdbcTemplate;

    @Override
    public People getPeopleById(Long id) {
        String sql = "select * from people where id = ?";
        return mainCamelJdbcTemplate.iQueryBean(sql, People.class, id);
    }
}

Advanced topics:

Two URL parameters are supported. The standard way and the non-standard way.

Standard configuration, like the following simple configuration: DatabaseConnection:Port:databaseName, the Camel framework will then generate standard connection strings to simplify configuration:

camel.mysql.main.url=127.0.0.1:3306:dbname

Non-standard way, if you want to customize your own more flexible URL connection,
Please follow the standard configuration of the mysql driver(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-configuration-properties.html), as:

camel.mysql.main.url=jdbc:mysql://127.0.0.1:3306/x?useUnicode=true&characterEncoding=utf-8

What data sources are supported? Default is Tomcat datasource. If you want to change the data source, set dataSourceType. It is recommended that a project have a fixed type of data source for easy maintenance. Set it to Hikari.

camel.mysql.main.dataSourceType=com.zaxxer.hikari.HikariDataSource
What parameters are supported? The default parameter in the above sample is the Tomcat data source configuration.
For details about the configuration parameters supported by Tomcat, see：`org.apache.tomcat.jdbc.pool.PoolConfiguration`，
See Class for the configuration parameters supported by Hikari: `com.zaxxer.hikari.HikariConfig`(The field of Hikari does not correspond exactly to the Set method. The name of the Set method prevails)
Like the method in Hikari`public void setMaximumPoolSize(int maxPoolSize)`，Use 'maximumPoolSize' when setting

Are transaction beans supported? This function is supported but disabled by default. If necessary, configure the transaction property, for example：

camel.mysql.main.transaction=true
The following beans are regenerated:       
mainMysqlTransactionManager, mainMysqlTransactionTemplate

Other parameters are described. Under standard URL configuration conditions, the framework's default encoding for the database is UTF-8 (recommended). If you have special requirements, you can set the characterEncoding parameter, such as:
```
camel.mysql.main.characterEncoding=UTF-8
```

1.2.2、 Redis usage instructions

The following is a sample configuration for Redis：

camel.redis.query.hostName=127.0.0.1
camel.redis.query.port=6694
camel.redis.query.password=Youcai
camel.redis.query.poolConfig.testOnBorrow=true
camel.redis.query.poolConfig.testWhileIdle=true
camel.redis.query.poolConfig.minIdle=20
camel.redis.query.poolConfig.maxIdle=100
camel.redis.query.poolConfig.maxTotal=100
camel.redis.query.poolConfig.maxWaitMillis=100

The keyword configured above is Query, which will generate the following beans, which can be injected and used according to the actual situation：

    queryRedisDataSource, queryRedisTemplate, queryCamelRedisTemplate

Using the sample procedure below, it is recommended to inject queryCamelRedisTemplate to use, such as:

@Repository
public class RankImpl implements RankDao {

    @Autowired
    @Qualifier("queryCamelRedisTemplate")
    RedisTemplate<String, String> queryCamelRedisTemplate;

    final String rankKey = "rank";

    @Override
    public void updateRank(long uid, long score) {        
        queryCamelRedisTemplate.opsForZSet().add(rankKey, uid, score);
    }
}

For more advanced configuration, see JedisPoolConfig.

1.2.3、 Kafka usage instructions

Kafka configuration is divided into producer and consumer, which are discussed separately below.

1.2.3.1、 Kafka Producer

Here is a sample producer configuration:

camel.kafka.producer.rank.topics=rank
camel.kafka.producer.rank.bootstrap.servers=127.0.0.1:9092
camel.kafka.producer.rank.acks=-1
camel.kafka.producer.rank.retries=3
camel.kafka.producer.rank.batch.size=100000
camel.kafka.producer.rank.client.id=sail_rank

The keyword configured above is rank. The following beans will be generated based on this keyword, which can be injected and used according to the actual situation:

rankKafkaProducer, rankCamelKafkaProducer

The sample program is as follows. It is recommended that injection rankCamelKafkaProducer be used, such as:

@Component
@Slf4j
public class DemoProducerDaoImpl implements DemoProducerDao{

    @Autowired
    @Qualifier("rankCamelKafkaProducer")
    private CamelKafkaProducer<String, String> rankCamelKafkaProducer;

    public void send() {        
        String message = "{\"uid\":\"3452378\",\"score\":3500,\"mark\":\"gift\"}";
        ProducerRecord<String, String> record = new ProducerRecord<>(rankCamelKafkaProducer.getTopics().get(0), UUID.randomUUID().toString(), message);
        rankCamelKafkaProducer.send(record, (metadata, e) -> {
            if (e != null) {
                log.error("", e);
            }
        });        
    }
}

Read the class ProducerConfig for more configurable parameters.

1.2.3.2、 Kafka Consumer

Here's a sample configuration for consumers:

camel.kafka.consumer.weekRank.bootstrap.servers=10.10.10.1:9092,10.10.10.2:9092,10.10.10.3:9092
camel.kafka.consumer.weekRank.group.id=week_rank_consume_local
camel.kafka.consumer.weekRank.client.id=week_rank_consume_local
camel.kafka.consumer.weekRank.security.protocol=SASL_PLAINTEXT
camel.kafka.consumer.weekRank.sasl.mechanism=PLAIN
camel.kafka.consumer.weekRank.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="Youcai" password="demo";
camel.kafka.consumer.weekRank.enable.auto.commit=true
camel.kafka.consumer.weekRank.auto.commit.interval.ms=1000
camel.kafka.consumer.weekRank.max.poll.records=1000

The keyword configured above is weekRank. This keyword will generate the following beans, which can be injected and used as required:

weekRankKafkaConsumer, weekRankCamelKafkaConsumer

Use the program samples as follows, it is suggested that injection weekRankCamelKafkaConsumer use, such as:

@Component
@Slf4j
public class DemoConsumer {

    @Autowired
    @Qualifier("weekRankCamelKafkaConsumer")
    CamelKafkaConsumer<String, String> weekRankCamelKafkaConsumer;

    @PostConstruct
    public void execute() {
        new Thread(() -> {
            weekRankCamelKafkaConsumer.subscribe(weekRankCamelKafkaConsumer.getTopics());
            while (true) {
                try{
                    ConsumerRecords<String, String> records = weekRankCamelKafkaConsumer.poll(Duration.ofMillis(1000));
                    records.forEach(record -> {
                        String item = record.value();
                        log.info("consumer: {}", item);
                    });
                } catch (Exception e) {
                    log.error("", e);
                }
            }
        }).start();
    }
}

The configurable parameters can be read in the class ConsumerConfig. If Kafka consumes too much, consider using ForkJoin to consume messages. See Exmaple module DemoForkJoinConsumer in detail, this code function is very good, our project is basically using this way to consume.

1.2.4、 Elasticsearch usage instructions

The following is a sample configuration file for ElasticSearch:

camel.elasticsearch.search.url=http://127.0.0.1:9200
camel.elasticsearch.search.username=x
camel.elasticsearch.search.password=x
camel.elasticsearch.search.maxConnTotal=15
camel.elasticsearch.search.maxConnPerRoute=35

In addition to defining the above configuration, you should also define the following code in the project root, pom.xml, otherwise you will use the version of elasticsearch7.x, If your project is 7.x, it is also highly recommended to define SpringBoot code with the same version number predefined.

<elasticsearch.version>6.2.2</elasticsearch.version>

The keyword configured above is search, which will be used to generate the following beans, which can be injected and used as required:

searchEsHighLevelClient

For configurable parameters, see Class CamelESPooling.

Use the sample program as follows to inject searchEsHighLevelClient using, as:

@Service
public class SearchServiceImpl implements SearchService{

    @Autowired
    @Qualifier("searchTestEsHighLevelClient")
    RestHighLevelClient searchEsHighLevelClient;

    @Override
    public SearchResponse search(String keyword) {
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchQuery("name", keyword)).size(10);
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.source(searchSourceBuilder);

        try {
            return searchEsHighLevelClient.search(searchRequest);
        } catch (IOException e) {
            e.printStackTrace();
            return null;
        }
    }
}

Appendix

Spring Boot Reference Documentation：

https://docs.spring.io/spring-boot/docs/2.5.9/reference/html/

How-to Guides：

https://docs.spring.io/spring-boot/docs/2.5.9/reference/html/howto.html#howto.data-access

Core Technologies：

https://docs.spring.io/spring-framework/docs/current/reference/html/core.html#beans-factory-nature

Spring Data JDBC - Reference Documentation：

https://docs.spring.io/spring-data/jdbc/docs/2.2.8/reference/html/#jdbc.repositories