function readOnly(count){ }
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
+ Start a Discussion
ASmithASmith 

Carriage Returns in Data Exports?

I performed a data export from the app using the UTF-8 encoding and found that my carriage returns in the Billing Address field have been converted into spaces (I haven't checked the other fields). I followed the special instructions for importing this type of file into Excel with no success.

Is there something I am missing here or is this a bug with the data exporter in the app?
ScotScot

Ouch.  Something has changed, I have this problem as well.

All my exports prior to November 2004 had the carriage return character properly retained (the equivalent of an Alt-CR in Excel).  My recent exports show the same problem you described.

I've not paid attention to the encoding format, using what the UI gave me as a default. So it is possible that the trouble is a change to the default format.

This is hard to experiment with when you can do one export a week.  Does anyone know about:

  1. Which encoding would be recommended
  2. Whether/what has changed

Thanks... Scot

((It appears that something changed with the last release. My 8-Nov-04 dump is fine, my 23-Nov-04 one - and all that followed - are not))

Message Edited by Scot on 05-31-2005 12:06 PM

ScotScot

This is looking like a bug to me ...
My exports recently have been with ISO-8859-1 format, and they show the same problem.

I called support; they are investigating. I'll comment when I get a response ...

Scot

ASmithASmith
Thank you Scot for checking into this. Glad to hear I'm not the only one who is experiencing the issue.

Andrew
ScotScot

On my part, I'm glad you noticed it. I've now got months of backups (exports) that have lost the formatting in their text fields.

Unfortunately, I've had no word back yet on the problem report.

ScotScot

Warning:
This behavior is a deliberate change, and there are no plans to fix it.

Results:
     Starting in November of last year,
     >  Any line breaks in fields are discarded from the weekly backups
     >  All street address lines are combined into a single line.

I don't know what this means to the rest of you, but to me, it means I no longer have a weekly backup of our data.


The first response from support:
      According to my QA engineer, this has always been the case.  Because this is a CSV format, they are separated by quotes and will not recognize any line breaks.  Line breaks can only be done with excel formatted files.
This was not correct, of course, because they can - and did prior to the Winter 05 release.

Second response:
      It turns out that the line breaks were removed because they were causing errors and problem when it came to importing them.  Unfortunately, they will not be placed back into the csv files.  I apologize for any inconvenience this may cause you.

I find this difficult to accept, and plan on pursuing it further through non-support paths.

ASmithASmith
Wow - that's an adventure you've been on to say the least. I'll keep working it from this end. I completely agree with you that the value of a backup is reduced if they are going to alter the data. It just doesn't make sense to me. Were you able to identify the PM for this area? Thanks for all the great info.

Andrew
darozdaroz
Wow... I'm very happy I haven't used Data Export in about 6 months for anything but testing. At the same time if I didn't have API access and I were a professional subscriber I'd be raising all holy hell right now.

For what it's worth, I'm with you on this one. Any export that's considered a 'backup' should under no circumstances manipulate or change the data. Period.
ScotScot
Not yet (PM), but I have a call in to my success and account managers....
Doug ChasmanDoug Chasman
I am the developer responsible for Weekly Export and wanted to let everyone on this thread know that the change was made to the handling of carriage returns in the Nov 2004 release to address issues with importing CSVs into a number of target environments. This is one of those situations where "fixing" that was a good thing for some of our customers and apparently undesirable for others. I apologize for any disruption that this has caused and wanted to let you know that we are now actively looking into this. My current thinking is that we need to add an option along the lines of Preserve CRs/LFs to the export request (just removing the conversion code would be an issue for the customers that requested this change in the first place). Thoughts/comments?
darozdaroz
Thanks for replying Doug.

I think this would be an excellent addition, and should solve everyone's problems with the export. However I'd have to advocate for the inverse of the option you suggest. Instead of giving the option to "preserve" the CR/LFs give an option to "remove" them instead.

The connotation associated with having to have an affirmative action to "preserve" something is that you are 'losing' or 'altering' something by default. One could carry that train of thought forward and ask what else is being altered? By requiring an action to "alter" the output, the 'bad' connotation is avoided.

My $.02
ASmithASmith

Hi Doug,

Makes complete sense to me.  I like your solution to the issue.  Any details on when we might be able to see this fix (please don't say future release).  We run backups on a bi-weekly basis and it is imperative that these have unaltered data.  Thank you for tackling this for us!

Andrew

andrew.smith@protiviti.com

 

ASmithASmith
Even better!  I like your thought process daroz.
Doug ChasmanDoug Chasman
I am working w/ my product manager now to see about getting the proposed fix in asap and should have an eta for you shortly.
ScotScot

Thanks, Doug - good response.

I agree with daroz as well - for several reasons:
    1)  as he stated, you avoid the "Classic Coke" problem
    2)  existing users do not have to be aware that the default operations changed
    3)  new users would get the CR/LF's included by default.

For those who count on the backups as a worst-case recovery device, the lack of the CR/LF's can cause a real problem ... but they're unlikely to notice this until the need it! And again, files with the CR/LF's embedded can be converted to ones without, but there's no way to go the other direction.

 

 

benjasikbenjasik
Thanks for all your help guys!
Doug ChasmanDoug Chasman
FYI - we will be releasing a fix for this in an upcoming patch (sorry, I just missed this week's patch cutoff). I have added a new "Replace carriage returns with spaces" option to the schedule export page, the default is unselected to preserve the current behavior.
ScotScot

   

Thanks for the quick response.

ScotScot

How are we coming on this patch??

Doug ChasmanDoug Chasman
Changes will be in the next patch release. I believe you will get it on this Friday.