How to convert newlines (replace \r\n with \n) across all varchar and nvarchar fields in a database

Question

I am recovering from a bug in a system I built where I did not take into account that IE generates Windows-style newlines (\r\n) and other browsers generate Unix-style newlines (\n) when posting HTML forms with text areas. Now I need to convert all Windows-style newlines (\r\n) to Unix-style newlines (\n) throughout the varchar and nvarchar fields in my SQL-Server database.

Is there a way to iterate through all tables/rows in T-SQL and replace instances of '\r\n' with '\n' for varchar and nvarchar fields?

EDIT: I think the replace part would be something like

REPLACE(@fieldContents, CHAR(13)+CHAR(10), CHAR(10))

The hard part is doing this across all varchar and nvarchar fields.

Answer 1

You could iterate through the system views in INFORMATION_SCHEMA and run dynamic SQL to do it. The relevant view should be INFORMATION_SCHEMA.COLUMNS.

A better approach is probably to have your UI deal with it when it has to display the values. Do you have a method to prevent values like that from getting into the DB in the future?

Here's some sample code that should get you started:

DECLARE
    @table_schema SYSNAME,
    @table_name   SYSNAME,
    @column_name  SYSNAME,
    @cmd          VARCHAR(MAX)

DECLARE cur_string_columns AS
    SELECT
        TABLE_SCHEMA,
        TABLE_NAME,
        COLUMN_NAME
    FROM
        INFORMATION_SCHEMA.COLUMNS
    WHERE
        DATA_TYPE IN ('VARCHAR', 'CHAR') AND  -- NVARCHAR and NCHAR?
        CHARACTER_MAXIMUM_LENGTH > 1

OPEN cur_string_columns

FETCH NEXT FROM cur_string_columns INTO @table_schema, @table_name, @column_name

WHILE (@@FETCH_STATUS = 0)
BEGIN
    SELECT @cmd = 'UPDATE
    ' + QUOTENAME(@table_schema) + '.' + QUOTENAME(@table_name) + '
SET ' + QUOTENAME(@column_name) + ' = REPLACE(' + QUOTENAME(@column_name) + ', CHAR(13) + CHAR(10), CHAR(10))'

    EXEC(@cmd)

    FETCH NEXT FROM cur_string_columns INTO @table_schema, @table_name, @column_name
END

CLOSE cur_string_columns

DEALLOCATE cur_string_columns

If you have large tables, this could take a LONG time to run. Also, optimally you would only update each table once, while this will update it once for each string column in the table. If I were doing this on a large database then I would change the script to account for that - order your cursor by the table schema and table name, append to the SET part of the string for each column in the table, only EXEC(@cmd) when the table changes and then reset your SET string.

Answer 2

Something like this? You could then dynamically execute these strings or just cut/paste the results and execute them in a query window.

select 'update ' + sc.name + '.' + t.name + ' set ' + c.name + ' = replace(' + c.name + ', CHAR(13)+CHAR(10), CHAR(10))'
from sys.columns c
    inner join sys.systypes st
        on c.system_type_id = st.xtype
            and CHARINDEX('varchar', st.name) <> 0
    inner join sys.tables t
        on c.object_id = t.object_id
    inner join sys.schemas sc
        on t.schema_id = sc.schema_id