[Voltron]

We get .csv files from companies we acquire to bring in their customers, their contacts and their software licenses (custom record). It may come from SalesForce, manually managed .xls files, Quicken, et al. Without fail, the import process will puke after selecting the file to import due to some number of lines (sometimes hundreds) that have characters it doesn't like for some reason. Often these look like normal American alphabet characters that I can actually fix by simply re-typing (copy/paste won't work). Other times, it's from the extended-ASCII character set. Spanish characters, German, French, other. It doesn't like those. They'll look totally "normal" in Notepad or Excel, but NS pukes. Takes forever to clean this, which usually has to involve changing something like ñ to n, which is an entirely different character.

I've tried different encodings, saving to Notepad as text then back to CSV. What should take maybe 20 minutes to prepare has become several hours and results in inaccurate data.

Currently, Netsuite Support has no reliable solution. They even seem surprised we're having the trouble. I've had these issues since first using NS 5 years ago. At my previous company, it was small files with maybe 500 lines. Here, I'm importing sometimes 75,000 lines. Other software seems to have no trouble.
This is a cached copy. Click here to see the original post.

[Ianic.Brisson]

I use Notepad++ which has the capability to change the character encoding. UTF-8 is usually a safe bet for encoding but you need to select it in the character encoding drop down when importing your CSV file in NetSuite.

Notepad++ is also a good tool for searching and replacing text strings so if there are still characters that don't get processed correctly by NetSuite you could clean them up. It also doesn't mess up the numeric data like Excel can.

Hope this helps

[Voltron]

RE: CSV Import needs major help

Originally posted by Ianic.Brisson

View Post

I use Notepad++ which has the capability to change the character encoding. UTF-8 is usually a safe bet for encoding but you need to select it in the character encoding drop down when importing your CSV file in NetSuite.

Notepad++ is also a good tool for searching and replacing text strings so if there are still characters that don't get processed correctly by NetSuite you could clean them up. It also doesn't mess up the numeric data like Excel can.

Hope this helps

Thanks. It would help except I've actually tried that. I use Notepad++ quite a bit. Find/replace helps for some things, but, as mentioned, some of those things either shouldn't have to be replaced (because Netsuite fields can hold those characters) or it just looks like regular characters.

[AGMNS]

RE: CSV Import needs major help

Voltron, I'm not sure if this is the same problem we have had, however I covered a solution in another post. When doing the CSV import choose Windows 1252 as the charachter encoding in the first screen of the CSV import wizard. Here's the link to my original post.

https://usergroup.netsuite.com/users…t=agmns&page=2

[JCirocco]

not sure if this will help but we originally had the same issue but only when we opened the files first in excel. We would receive a CSV file with quotes around the chaacter fields. When saved from Excel, the quotes were removed. Now I force quotes around every field and that solved our problem.

[capitoladmin]

First of all, there is a line limit on the CSV imports of 25000 rows. Break the import job up into multiple files no more than 250000 rows each.

To attempt to resolve the character issue, saved the Excel file as CSV.

Open the file with notepad.

Check that saving as CSV did not convert any of the characters to question marks (?).

Perform a File > Save As choose Encoding method as UTF-8

Import via csv, but set character encoding as Unicode (UTF-8)

Tips to ensure data integrity:

1. always perform update on a few records first to make sure a given process works.

2. always proof data after import, even if no errors

I have successfully imported many characters with the above procedure that do not work with the Western (Windows 1252) character encoding option, including the funky characters that are not actually alphabetic characters, rather some extended-ASCII or unicode look alikes.

This incorrect encoding caused some of our item #'s and descriptions to go awry on imports, so was a huge problem of cleaning data after Netsuite's data migration team initial import. I have been scrubbing data off and on for a year to clean up the 70K+ initial SKU import. It took a while and several calls to figure this out, after being told that certain symbols could NOT be imported to Netsuite – ohm, alpha, lambda, etc.

Happy imports!

[jchernia]

RE: CSV Import needs major help

In terms of getting non-US excel files, one solution I've suggested before (since we don't support country specific character encodings), is to upload the file to Google Docs as Excel and then export as CSV. This has the downside of having to go through Google obviously.

If you are only getting CSV files and they've already been exported in a non-supported character set (eg 1250 instead of 1252), then either NetSuite would have to support the code page or you'd have to find some sort of CodePage Converter (Google Code Page File Converter for examples).

What other file sources (besides Excel) are giving you problems?

-John

[corey]

I recently ran into this issue with an import from Salesforce. I found that =clean(#) worked like a charm, at least for the weirdness that SF throws into a text field.

[Author]

Posts