Some tips to make you look like a pro!
Here at Tamarack, we work with data a lot!
Like you, we are uploading and extracting data on a daily basis for process automation, reporting, and business intelligence. Given the plethora of new on-premise and cloud-based applications, we are seeing increasing demand for Data Migration support. As we have expertise in so many applications, we are often engaged to help migrate data between these platforms.
Many of the data migration issues can be traced to one of two issues:
- “Hidden” or non-displayable characters
- Incorrect End of Line (EOL) characters
Both of these issues can be easily exposed with a free open-source application: Notepad++
*If you are running on IOS, we suggest you search for “Best alternatives for Notepad++ for Mac”.
Exposing Hidden Characters
After opening the data file in Notepad++, go to the View options and under “Show Symbols” to reveal all hidden characters. This option will highlight them in black. It will even add an arrow symbol when there is a “tab” which helps when working with tab delimited files.
If you would like to learn more about these offending characters, do a quick search on ASCII – American Standard Code for Information Interchange: www.AsciiTable.com
If you move files between Windows and Linux/Unix systems, you’ve likely been tripped up on the incorrect End of Line character.
In the Microsoft world, an End of Line is designated by Carriage Return/Line Feed (CR/LF) characters, while Linux systems simply use the Line Feed (LF). Once again, Notepad++ will expose these characters for you.
But before you start deleting the offending CR or adding it in, there is an easier way! Just search the documentation of the application you are using for "Transfer Settings." Under “text” mode, the conversion between CR/LF and LF will happen automatically!