Geeks With Blogs
El Grego BizTalk blog
Or how to fix a bug while introducing another one that's a lot nastier. Let's start by describing the original bug: the BizTalk flatfile assembler has issues with custom target (output) encodings.

When compiling a custom pipeline with the FF assembler and configuring the 'target charset' in the pipeline designer everything works as expected and you will get your messages in the desired encoding.

When you want to dynamically control the encoding, according to the docs, you should also be able to do this by writing the XMLNORM.TargetCharSet property onto the message context.

In my case the desired output was 'Windows-1252'.

I was able to verify that this technique indeed works using a default XmlTransmit assembler pipeline.
With a custom FF assembling pipeline though I got a question mark in the flatfile output for every special character.
Since the only differentiator being the pipeline I am quite confident that it's a FFASM assembler bug.

Now to come to the second problem: apparently this bug was already discovered over a year ago and I guess it was never reported to Microsoft (since it is still not fixed in BTS 2009, the RC I checked). It might be a regression since people reported that it only happens from version 2006 R2.

Instead, the author decided the shortest path to success was to develop a custom encoding pipeline component that takes care of the output encoding. The issue showed up a few more times since then and it looks like other people were inspired by his workaround.

Now take a good look again at how the author implemented the custom encoding. There is something terribly wrong with it.

HINT: the input encoding is UTF-8, a variable byte length encoding. Inside the loop a fixed number of bytes is read from the input. Got it? Posted on Monday, August 3, 2009 4:54 AM BizTalk - EAI - B2B | Back to top

Comments on this post: Question marks in your flatfile output?

# re: Question marks in your flatfile output?
Requesting Gravatar...
Ok, but you did not answer how to solve the problem other than pointing out obvious flaws in others. Check this one out though:
Left by Zsanett on Nov 25, 2011 6:05 PM

# re: Question marks in your flatfile output?
Requesting Gravatar...
Zsanett, check the next post from 2009 for the solution. Sorry for late reply :)
Left by Grego on Oct 26, 2012 8:25 AM

Your comment:
 (will show your gravatar)

Copyright © Gregory Van de Wiele | Powered by: