MonitorWare Knowledge Base

Hi,
I have some non-English edition windows 2003 servers and some English edition .and I use rsyslog to receive logs , now the English edition's log can be written to mysql db correctly and the non-english edtion not.the non english messages were stored like "......????????...." . how to fix it?

sorry for my English.

Thanks.

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

what is the non-english - western or asian character sets? Which application sends the syslog messages from Windows?

asian character set. I have tried evtsys ,snare and evtrpt , the problem is same.I am running rsyslog 3.20.4

that's what I thought. Could you try http://www.eventreporter.com (it's commercial, but there is a 30 day trial). If that does not work, I know much better where to look at.

sorry for the long time replay.I actually use eventreporter to send the logs,but it did not fix the issue,i take a screenshot. and I found the article http://www.rsyslog.com/doc-syslog_protocol.html on this website

character encoding in MSG: is is problematic to do the right UTF-8 encoding. The reason is that we pick up the MSG from the local domain socket (which got it from the syslog(3) API). The text obtained does not include any encoding information, but it does include non US-ASCII characters. It may also include any other encoding. Other than by guessing based on the provided text, I have no way to find out what it is. In order to make the syslogd do anything useful, I have now simply taken the message as is and stuffed it into the MSG part. Please note that I think this will be a route that other implementors would take, too.

does it means i need to change the phplogcon config to connect the mysql database with the same charset as the message is ? And how to config that?

I need to discuss this with Andre, the phpLogCon main developer. Hopefully he has a suggestion. It may also be necessary to do changes to rsyslog. I am willing to do, but in any case we need someone with a test environment. Would you be willing to run a couple of tests during this effort (no promise on how long it will take).

Rainer

No problem ,I actually run the rsyslog on a virtual machine

The UTF8 Encoding which is available in EventReporter 10.0 or 10.1 is not compatible with other Syslog Servers, only with WinSyslog and MonitorWare Agent. We are not using RFC 5424 in EventReporter yet, which will solve this problem once and for all.

So you cannot use rsyslog to receive UTF8 encoded messages from EventReporter at the moment.

best regards,
Andre Lorbach

EDIT: I discussed this issue with Rainer, we may can get this running the one or the other way. But we will need to perform some more tests first.

Andre was a bit distorted by what he is currently working on Actually, if you use traditional (non-UTF8) encodings, things should work (at least I know from other people it worked for them). I had a discussion with Andre, but he is too overwhelmed with work at the moment to dig deeper. So it looks like we need to discuss this next week. The good thing is that Andre seems to have something that works well enough at a testbed. Plus, I'll re-check what it takes to support Unicode internally, but I guess this would be a very (aka "too") big change for the rsyslog engine to be done quickly (that's a drawback from forking sysklogd, we had all 8 bit chars...).

Rainer

Unfortunately,it doesn't work for me I send logs with the system default chartset(gbk,a 8 bit chars),but it is same. I googled some php articles point out that it should be set a php meta like

Code: Select all

<meta http-equiv="Content-Type" content= "text/html"; charset=gbk" />

and set a connect charset before operating db like

Code: Select all

mysql_query("SET NAMES GBK");

but i have very limited php knowledges, can Andre help me get the right spot should be modified?

sorry,gbk is a double-byte character set

i think i have found a solution.
i use utf8 for rsyslog and phplogcon
rsyslog need change:
plugins/ommysql/ommysql.c:162 add

Code: Select all

      if(mysql_query(pData->f_hmysql,"SET NAMES utf8")){
         /* error occured, try to re-init connection and retry */
         reportDBError(pData, bSilent);
         closeMySQL(pData); /* ignore any error we may get */
         iRet = RS_RET_SUSPENDED;
      }


phplogcon need change:
change:

Code: Select all

include/functions_common.php:1028:   return htmlentities($myStr, ENT_COMPAT, "UTF-8");

add:

Code: Select all

classes/logstreamdb.class.php:166:      mysql_query("SET NAMES utf8", $this->_dbhandle);