Mfc Font binding and Rich Edit Control RICHEDIT50W does not display Unicode Properly

Question

Latest update:

Well, I found a culprit of some sort. I changed the control to RichEdit20W from the 50W and it displays the Hangul (Korean) now. I did not have to change any other code except for the init, added AfxInitRichEdit2(), and commented out the LoadLibrary(L"MsftEdit.dll"). The AfxInitRichEdit5() is not available for VS2010. All things being equal and Rich Edit 4.1 has been available since VS2005, it should have worked. I can't upgrade to VS2015 right now so I'm stuck. I'm going to offer a bounty though for anybody who can make the Hangul work with 50W and VS2010.

---

I have a dilemma that I can't seem to solve.

I have an mfc Unicode app that uses CEdit and CRicheditCtrl.
The Rich Edit is 50W loaded from MsftEdit.dll and verified with Spy++
that the class name is RICHEDIT50W.

My problem:

I'm using the same Font Courier New for both the CEdit and CRichEditCtrl.

As a test, I used some of the Hangul symbols to see the output for both
controls.

CEdit outputs ᄀᄁᄂᄃᄄᄅᄆᄇᄈ
while the
CRichEditCtrl outputs a box for each character, like there is no glyph for it.

If they are using the same font, shouldn't I see the same output characters?

I think that font-binding is not a problem, both have the same default font.

Can anybody solve this riddle ?

Thanks in advance!

Note that this happens with some other character sets as well, not just Hangul

---

Update

I looked at the VS2010 WordPad example, it uses CRichEditView but it
provides wrappers to access the embedded CRichEditCtrl.

I thought I could glean some info but I can't see how they are doing the
Rich Edit control calls.

This is how I am generating the font's for both controls.
But, I'm showing specifically the Rich Edit part.

The doc's say that Font binding should handle switching from the default
font to the font at the current insertion point.

I am doing insertion mostly at the end using
ctrl.SetSel(-1,-1);
ctrl.ReplaceSel( str );

And, according to the doc's, this should change to the correct font as needed,
if other than the default font.

In the WordPad sample, if I paste in the Hangul text, the font switches to
Gulim.

Here is my code:

LOGFONT lf;
int pitch = 10;
memset(&lf, 0, sizeof(LOGFONT));

HDC hdc = ::GetDC(NULL);
lf.lfHeight = -MulDiv( pitch, GetDeviceCaps(hdc, LOGPIXELSY), 72);
lf.lfPitchAndFamily = DEFAULT_PITCH | FF_DONTCARE;
lstrcpy(lf.lfFaceName, _T("Courier New") );
lf.lfWeight = FW_NORMAL;
lf.lfCharSet = ANSI_CHARSET;  // English, but use DEFAULT_CHARSET if not
lf.lfQuality = DEFAULT_QUALITY;


if ( !m_Font.CreateFontIndirect(&lf) )
{   // Ours didn't work, create a system default fixed font
    // ( Ignore, example for post only. Never gets called though )
    //memset(&lf, 0, sizeof(LOGFONT));
    //::GetObject(GetStockObject(ANSI_FIXED_FONT), sizeof(LOGFONT), &lf);
    //m_Font.CreateFontIndirect(&lf);
}


// Save the generated Font LOGFONT 
m_lf = lf;

// Set the default Font CHARFORMAT2
memset( &m_cfDefaultFont, 0, sizeof(m_cfDefaultFont) );
m_cfDefaultFont.cbSize = sizeof(m_cfDefaultFont);
m_cfDefaultFont.dwMask = CFM_CHARSET | CFM_FACE | CFM_WEIGHT ;
m_cfDefaultFont.bCharSet = m_lf.lfCharSet;
lstrcpy( m_cfDefaultFont.szFaceName, m_lf.lfFaceName );
m_cfDefaultFont.wWeight = m_lf.lfWeight;

// Finally set the font in the controls
m_RichEdit.SetFont( &m_Font );

// Same effect as m_RichEdit.SetFont()
//m_RichEdit.SendMessage(EM_SETCHARFORMAT, SCF_ALL, &m_cfDefaultFont);

// This displays nothing but 'box' glyphs
m_RichEdit.SetWindowTextW(_T("ᄃᄄᄅᄆᄇᄈᄉᄊᄋᄌᄍᄎ"));

Update 2

This is how I initialize the Rich Edit in the app.
And shows the usage of 50W in a dialog control.

-- winapp.cpp
BOOL CMyApp::InitInstance()
{
    // ...... //

    CString strRichEdit = _T("Msftedit.dll");
    m_hMsfteditDll = AfxLoadLibrary( strRichEdit );
    if ( m_hMsfteditDll == NULL )
    {
        CString str;
        str.Format(_T("Error: Cannot find Rich Edit component %s"), strRichEdit );
        AfxMessageBox(str);
        return FALSE;
    }
    return TRUE;
}
int CRegexFormatApp::ExitInstance() 
{
    if ( m_hMsfteditDll != NULL )
        AfxFreeLibrary( m_hMsfteditDll );
    return CWinAppEx::ExitInstance();
}

// =========================

-- .rc 
CONTROL         "",IDC_RICH_EDIT,"RICHEDIT50W",WS_VSCROLL | WS_HSCROLL,40,15,148,28

-- Dlg.h
CRichEditCtrl m_RichEdit;

-- Dlg.cpp
void Dlg::DoDataExchange(CDataExchange* pDX)
{
    CDialogEx::DoDataExchange(pDX);
    DDX_Control(pDX, IDC_RICH_EDIT, m_RichEdit); // Use .rc setting to Create/Attach
}

BOOL Dlg::OnInitDialog()
{
    CDialogEx::OnInitDialog();
    CreateAndSetRichFont();  // Code shown above
    m_RichEdit.SetWindowTextW( ... );
}

Sinking

Works just fine in Wordpad.exe, it also uses that control. You'll have to show us. – Franz

Ok, I have a delay, will have to show source in a while.. – Sinking

Can we see the code where you create and assign the font to either control? – Mowbray

Ok, added some code. – Sinking

I hope it's not a case where I have to test every character that get's inserted for that char's lcid and set the CHARFORMAT2 appropriate for the insertion on each series with the same lcid. I was hoping this could be done automatically with font binding like the doc's imply. – Sinking

I tried your code, it looks fine, except the font is a little small. Maybe there is probably a problem with loading MsftEdit.dll. Show the library initialization and Create function. VS2015 has AfxInitRichEdit5(), I don't know what VS2010 does. – Affinal

@BarmakShemirani - Added msftedit.dll initialization code. The only thing available to load a richedit code in VS2010 is AfxInitRichEdit2() so I have to load msftedit.dll via a AfxLoadLibrary() call. To create the control, I'm using a creation from a DDX_Control() call, that uses the .Rc settings via the ID. I'm quickly losing hope. – Sinking

I tried your code in VS2008, almost exactly the way you have, it works fine. Try creating a new dialog project to test this code. Maybe remove AfxFreeLibrary I don't what that's doing. Or try a different font. – Affinal

By the way, I noticed you have a bunch of things about ANSI. Change it default char set, remove references to ANSI. – Affinal

Answer 1

This code should work in a Unicode project:

BOOL CMyApp::InitInstance()
{
    CWinApp::InitInstance();
    LoadLibrary(L"MsftEdit.dll");
    ...
}

BOOL CMyDialog::OnInitDialog()
{
    CDialog::OnInitDialog();

    static CRichEditCtrl redit;
    CWnd *wnd = &redit;
    wnd->Create(MSFTEDIT_CLASS, L"ᄃᄄᄅᄆᄇᄈᄉᄊᄋᄌᄍᄎ",
            WS_CHILD | WS_VISIBLE, CRect(0,0,300,300), this, 1001, 0);
    ...
    //redit is not to be mixed up with controls created in dialog editor.
}

Edit ----------------

NONCLIENTMETRICS info = { sizeof(info) };
SystemParametersInfo(SPI_GETNONCLIENTMETRICS, sizeof(info), &info, 0);
LOGFONT logfont = info.lfMessageFont;

//CClientDC has automatic cleanup, use it instead of GetDC
CClientDC dc(this);
logfont.lfHeight = -MulDiv(abs(logfont.lfHeight), dc.GetDeviceCaps(LOGPIXELSY), 76);

CFont font;
font.CreateFontIndirect(&logfont);
m_RichEdit.SetFont(&font);

m_RichEdit.SetWindowText(L"ᄃᄄᄅᄆᄇᄈᄉᄊᄋᄌᄍᄎ");

m_RichEdit.SetSel(1, 1);

CString test = L"(Test, ελληνικά)";

//Test to make sure we can see Unicode text
AfxMessageBox(test);
m_RichEdit.ReplaceSel(test);

//optional:
//get default CHARFORMAT
CHARFORMAT2 charFormat;
//m_RichEdit.GetSelectionCharFormat(charFormat);
m_RichEdit.GetDefaultCharFormat(charFormat);

Affinal

I think I found something. If I copy (ctrl-c) the Hangul text from Notepad or CEdit and paste into the Rich ctrl, I get boxes. If I copy from MS Word (ie. another Rich ctrl) and paste, I get Hangul. ?? Wtf ??. Do you think this has to do with Multi-Byte or clipboard Unicode text type ? Also, I am mostly populating programmatically with SetWindowText or SetSel( -1,-1 );ReplaceSel( str ); where str is LPCTSTR (ie. wide chars). Also, when I copy the pasted box text from the rich control to another spot in the control, it comes up Hangul.. – Sinking

I think your project is not Unicode. So sometimes it works, sometimes it doesn't. You should change the project to Unicode if possible. If that's not an option, you can force in Unicode control in ANSI project (see edit). Note that MFC functions will always default to ANSI in ANSI project. So you would have to use WinAPI to access Unicode functions. You can't use CRichEdit::ReplaceSel to send Unicode text in ANSI project. Instead you have to use SendMessageW(hedit, EM_REPLACESEL...) it gets complicated... – Affinal

It's a Unicode project /D "_UNICODE", I thought ReplaceSel just sends EM_REPLACESEL to the control. I'll try that anyway and see what happens. Cutting and pasting Unicode between wordpad and notepad and MSWord has similar weird effects too. – Sinking

Nevermind about EM_REPLACESEL because it should be fine if you have Unicode project. Maybe it's a problem or the source-text is not Unicode. Before inserting text in to RichEdit, display it with message box to make sure it is Unicode text. See 2nd edit. – Affinal

Well, I found a culprit of some sort. I changed the control to RichEdit20W from the 50W and it displays the Hangul (Korean) now. I did not have to change any other code except for the init, added AfxInitRichEdit2(), and commented out the LoadLibrary(L"MsftEdit.dll"). The AfxInitRichEdit5() is not available for VS2010. All things being equal and Rich Edit 4.1 has been available since VS2005, it should have worked. I can't upgrade to VS2015 right now so I'm stuck. I'm going to offer a bounty though for anybody who can make the Hangul work with 50W and VS2010. – Sinking

I think I had tried your code in VS2008, it worked fine. – Affinal

The way you are writing your code it looks like UNICODE is not set. Can you confirm once again that the following code compiles: m_RichEdit.SetWindowText(L"X"); (enter exactly as I typed in) – Affinal

I'm using the wide characters. It's purely Unicode throughout. The real code is fairly complex, using hilighting and effects all over the place. I didn't need to change anything else. I think the Hangul and 50W and VS2010 combination could be the problem. There might be other languages that aren't correctly handled via Font Binding, I haven't checked yet. – Sinking

L"xx" is a Unicode (multi-byte) string, but I use strictly wide char set throughout. Examining WordPad for the 100th time, it uses 20A in the sample. I thought I'd give 20W a try. – Sinking

You didn't answer my question. Unicode is not the same thing as Multi-byte. I don't think our compilers have the same settings. – Affinal

Well, I have Use Unicode Character Set in the character set of Project Defaults. I convert to byte in some parts of the program as needed, but this is not relavent to this code. I tried the L"notation" as well as the _T("\uXXXX\uYYYY") before, both had the same results, ie. squares. Wide character is Unicode (ie. UTF-16) with surrogate pairs as needed. – Sinking

Okay that's not the issue then. By the way, _T("text") is a macro. If you look at its definition, it is interpreted as L"text" when UNICODE is defined, otherwise it is "text". – Affinal

A literal that has a storage class of wchar_t*. I have that mapped as a native type. L".." is just general compiler notation for Unicode. – Sinking

Thanks for all your help ! – Sinking