Spring-Boot, Can't save unicode string in MySql using spring-data JPA

Question

I have my application.properties set up like this :

spring.datasource.username = root
spring.datasource.password = root
spring.datasource.url = jdbc:mysql://localhost:3306/dbname?useUnicode=yes&characterEncoding=UTF-8
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.jpa.show-sql = true
spring.jpa.hibernate.ddl-auto = update
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect

And In my pom.xmlI have property set up like this :

<properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <start-class>toyanathapi.Application</start-class>
        <java.version>1.8</java.version>
</properties>

My entity : @Entity public class DailyRashifalEntity {

@Id
@GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private String date;
private int rollno;
private String name;
//Constructors and getters/setters 
}

Problem 1: If I use the above setup I get the exception

java.sql.SQLException: Incorrect string value: '\xE0\xA4\xA7\xE0\xA4\xBE...

Problem 2 : If I change the datasource url into this :

spring.datasource.url = jdbc:mysql://localhost:3306/dbname

The unicodes in my database get saved like this

 29 | 2074-03-04 |        3 | ?????????????? ?????,?????? ??????, ??????????? ????? ? ???? ???? ???? ??????  

How can I save them in Mysql like they are in unicode instead of getting all the unicode data converted into ???????? .

Focus

drop the database and try again. u can set spring.jpa.hibernate.ddl-auto = create for testing. – Dicho

have you tried setting the charset on the connection as well ? – Ilsa

@premkumar no luck.. – Focus

@FMashiro I am starting spring in last few days and I am not quite sure what is it like that you are telling . Can you explain plase ? – Focus

I think your database schema character encoding is not utf8. Could you check it executing the query: SELECT default_character_set_name FROM information_schema.SCHEMATA WHERE schema_name = "schemaName"; – Domitiladomonic

nevermind, you already do it. Ehm, what's the collation on your table ? – Ilsa

sorry again, what is collation of table, where is it defined ? And I have updated my schema of @Entity can it be of some help for you to find whats wrong? – Focus

paste the output for show create table DailyRashifalEntity ; – Dicho

create table my_table (id bigint not null auto_increment, date varchar(255), rashi_id integer not null, rashifal varchar(255), primary key (id)) – Focus

paste output for show create database dbname – Dicho

try this CREATE DATABASE mydatabase CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; – Knack

No luck again... – Focus

Answer 1

In your /etc/mysql/my.cnf file change the following.

[mysql]
default-character-set=utf8
[mysqld]
character-set-server=utf8

Unscientific

Answer 2

Keep your hibernate configuration Like this

jdbc:mysql://localhost:3306/dbname?useUnicode=yes&characterEncoding=UTF-8

And Change your DB Collation Like this

ALTER TABLE tablename CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

More information : Link

Jurgen

My project had the same issue you have described. and this worked for me. – Jurgen

Problem was not in my project, rather it was in mysql config in Mechine – Focus

Answer 3

See "question marks" in https://mcmap.net/q/55219/-trouble-with-utf-8-characters-what-i-see-is-not-what-i-stored .

Also,

⚈  spring.jpa.properties.hibernate.connection.characterEncoding=utf-8 
⚈  spring.jpa.properties.hibernate.connection.CharSet=utf-8 
⚈  spring.jpa.properties.hibernate.connection.useUnicode=true 

Encomiastic

Answer 4

Solved it by applying to the column definition as shown below:

public class Goal{

@Column(name = "SOURCE_OF_FUNDS", length = 6000, columnDefinition = "VARCHAR(6000) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL")
private String sourceOfFunds;

}

Bandicoot

Answer 5

if you want save non-ascii character correctly, you should make sure:

• Created MySQL table with charset=encoding is UTF-8
• Connect MySQL table with charset=encoding is UTF-8

following is how to achieve it:

Create mysql table with utf-8 charset

method 1 (I use this method): config in Spring Boot JPA

after refer another Chinese post, my final working configuration:

• file: src/main/java/com/crifan/smartelectricserver/MySQL55DialectUtf8mb4.java
• code:

package com.crifan.smartelectricserver;

import org.hibernate.dialect.MySQL55Dialect;;
// import org.hibernate.dialect.MySQL5InnoDBDialect; // Deprecated

// public class MySQL5InnoDBDialectUtf8mb4 extends MySQL5InnoDBDialect {
public class MySQL55DialectUtf8mb4 extends MySQL55Dialect {
  @Override
  public String getTableTypeString() {
    return "ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE utf8mb4_unicode_ci";
  }
}

which set table charset to utf8mb4 and collation to utf8mb4_unicode_ci

and

• file: src/main/resources/application.properties
• config:

spring.jpa.database-platform=com.crifan.smartelectricserver.MySQL55DialectUtf8mb4

in which:

• com.crifan.smartelectricserver.MySQL55DialectUtf8mb4: corresponding above file com/crifan/smartelectricserver/MySQL55DialectUtf8mb4.java

method 2: set utf-8 in MySQL config file my.cnf

find your my.cnf in your machine, edit to:

[client] 
default-character-set = utf8mb4 
[mysql] 
default-character-set = utf8mb4 
[mysqld] 
character-set-client-handshake = FALSE 
character-set-server = utf8mb4 
collation-server = utf8mb4_unicode_ci 
init_connect='SET NAMES utf8mb4'

Connect MySQL with UTF-8

• file: src/main/resources/application.properties
• config:

spring.datasource.url=jdbc:mysql://${MYSQL_HOST:localhost}:3306/smart_electric?useUnicode=yes&characterEncoding=UTF-8

in which:

• useUnicode=yes&characterEncoding=UTF-8: means connect MySQL with charset utf-8

Illyes

Answer 6

Non of the answer worked for me … except the url encoding part.

The solution in my case is twofold:

1- Add encoding in the URL of Database config bean:

<property name="jdbcUrl" value="jdbc:mysql://localhost:3306/customersdb?useSSL=false&amp;useUnicode=yes&amp;characterEncoding=UTF-8&amp;characterSetResults=UTF-8" />

2- Add this configuration to your dispatcher config file:

<filter>
    <filter-name>encoding-filter</filter-name>
    <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
    <init-param>
        <param-name>encoding</param-name>
        <param-value>UTF-8</param-value>
    </init-param>
    <init-param>
        <param-name>forceEncoding</param-name>
        <param-value>true</param-value>
    </init-param>
</filter>

<filter-mapping>
    <filter-name>encoding-filter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
</filter-mapping>

Otherwise, other configurations don't take effect.

Pharos