Converter com eficiência linhas em colunas no servidor sql

387

Estou procurando uma maneira eficiente de converter linhas em colunas no SQL Server, ouvi dizer que o PIVOT não é muito rápido e preciso lidar com muitos registros.

Este é o meu exemplo:

   -------------------------------
   | Id | Value  | ColumnName    |
   -------------------------------
   | 1  | John   | FirstName     |
   | 2  | 2.4    | Amount        |
   | 3  | ZH1E4A | PostalCode    |
   | 4  | Fork   | LastName      |
   | 5  | 857685 | AccountNumber |
   -------------------------------

Este é o meu resultado:

---------------------------------------------------------------------
| FirstName  |Amount|   PostalCode   |   LastName  |  AccountNumber |
---------------------------------------------------------------------
| John       | 2.4  |   ZH1E4A       |   Fork      |  857685        |
---------------------------------------------------------------------

Como posso construir o resultado?

tbag
fonte

Respostas:

544

Existem várias maneiras de transformar dados de várias linhas em colunas.

Usando PIVOT

No SQL Server, você pode usar a PIVOTfunção para transformar os dados de linhas em colunas:

select Firstname, Amount, PostalCode, LastName, AccountNumber
from
(
  select value, columnname
  from yourtable
) d
pivot
(
  max(value)
  for columnname in (Firstname, Amount, PostalCode, LastName, AccountNumber)
) piv;

Veja demonstração .

Pivô com número desconhecido de columnnames

Se você tiver um número desconhecido do columnnamesqual deseja transpor, poderá usar o SQL dinâmico:

DECLARE @cols AS NVARCHAR(MAX),
    @query  AS NVARCHAR(MAX)

select @cols = STUFF((SELECT ',' + QUOTENAME(ColumnName) 
                    from yourtable
                    group by ColumnName, id
                    order by id
            FOR XML PATH(''), TYPE
            ).value('.', 'NVARCHAR(MAX)') 
        ,1,1,'')

set @query = N'SELECT ' + @cols + N' from 
             (
                select value, ColumnName
                from yourtable
            ) x
            pivot 
            (
                max(value)
                for ColumnName in (' + @cols + N')
            ) p '

exec sp_executesql @query;

Veja demonstração .

Usando uma função agregada

Se você não quiser usar a PIVOTfunção, poderá usar uma função agregada com uma CASEexpressão:

select
  max(case when columnname = 'FirstName' then value end) Firstname,
  max(case when columnname = 'Amount' then value end) Amount,
  max(case when columnname = 'PostalCode' then value end) PostalCode,
  max(case when columnname = 'LastName' then value end) LastName,
  max(case when columnname = 'AccountNumber' then value end) AccountNumber
from yourtable

Veja demonstração .

Usando várias junções

Isso também pode ser concluído usando várias junções, mas você precisará de alguma coluna para associar cada uma das linhas que você não possui em seus dados de amostra. Mas a sintaxe básica seria:

select fn.value as FirstName,
  a.value as Amount,
  pc.value as PostalCode,
  ln.value as LastName,
  an.value as AccountNumber
from yourtable fn
left join yourtable a
  on fn.somecol = a.somecol
  and a.columnname = 'Amount'
left join yourtable pc
  on fn.somecol = pc.somecol
  and pc.columnname = 'PostalCode'
left join yourtable ln
  on fn.somecol = ln.somecol
  and ln.columnname = 'LastName'
left join yourtable an
  on fn.somecol = an.somecol
  and an.columnname = 'AccountNumber'
where fn.columnname = 'Firstname'
Taryn
fonte
6
+1. . . Mas no último exemplo, você pode usar, cross joine não left joinporque cada subconsulta retorna uma linha.
Gordon Linoff
11
Preciso criar uma consulta dinâmica porque não sei o número de linhas. Vamos falar sobre a conversão de uma tabela com 10.000.000 milhões de registros
tbag
11
@tbag Se você tiver um número desconhecido de linhas, será necessário usar o sql dinâmico, mas saiba que transformar milhões de linhas não será eficiente.
Taryn
11
Não posso usar uma visualização para essa transformação. Que tal usar um TFV? é por isso que estou procurando uma solução eficiente.
tbag
4
@tbag dinâmico SQL não pode ser usado em uma exibição ou uma função com valor de tabela, deve ser utilizado em um procedimento armazenado
Taryn
11

Este é um método, e não apenas um script, mas oferece muito mais flexibilidade.

Primeiro de tudo Existem 3 objetos:

  1. Tipo de tabela definido pelo usuário [ColumnActionList ] -> mantém dados como parâmetro
  2. SP [proc_PivotPrepare ] -> prepara nossos dados
  3. SP [ proc_PivotExecute] -> executar o script

CREATE TYPE [dbo]. [ColumnActionList] AS TABLE ([ID] [smallint] NOT NULL, [ColumnName] nvarchar NOT NULL, [Action] nchar NOT NULL); VAI

    CREATE PROCEDURE [dbo].[proc_PivotPrepare] 
    (
    @DB_Name        nvarchar(128),
    @TableName      nvarchar(128)
    )
    AS
            SELECT @DB_Name = ISNULL(@DB_Name,db_name())
    DECLARE @SQL_Code nvarchar(max)

    DECLARE @MyTab TABLE (ID smallint identity(1,1), [Column_Name] nvarchar(128), [Type] nchar(1), [Set Action SQL] nvarchar(max));

    SELECT @SQL_Code        =   'SELECT [<| SQL_Code |>] = '' '' '
                                        + 'UNION ALL '
                                        + 'SELECT ''----------------------------------------------------------------------------------------------------'' '
                                        + 'UNION ALL '
                                        + 'SELECT ''-----| Declare user defined type [ID] / [ColumnName] / [PivotAction] '' '
                                        + 'UNION ALL '
                                        + 'SELECT ''----------------------------------------------------------------------------------------------------'' '
                                        + 'UNION ALL '
                                        + 'SELECT ''DECLARE @ColumnListWithActions ColumnActionList;'''
                                        + 'UNION ALL '
                                        + 'SELECT ''----------------------------------------------------------------------------------------------------'' '
                                        + 'UNION ALL '
                                        + 'SELECT ''-----| Set [PivotAction] (''''S'''' as default) to select dimentions and values '' '
                                        + 'UNION ALL '
                                        + 'SELECT ''-----|'''
                                        + 'UNION ALL '
                                        + 'SELECT ''-----| ''''S'''' = Stable column || ''''D'''' = Dimention column || ''''V'''' = Value column '' '
                                        + 'UNION ALL '
                                        + 'SELECT ''----------------------------------------------------------------------------------------------------'' '
                                        + 'UNION ALL '
                                        + 'SELECT ''INSERT INTO  @ColumnListWithActions VALUES ('' + CAST( ROW_NUMBER() OVER (ORDER BY [NAME]) as nvarchar(10)) + '', '' + '''''''' + [NAME] + ''''''''+ '', ''''S'''');'''
                                        + 'FROM [' + @DB_Name + '].sys.columns  '
                                        + 'WHERE object_id = object_id(''[' + @DB_Name + ']..[' + @TableName + ']'') '
                                        + 'UNION ALL '
                                        + 'SELECT ''----------------------------------------------------------------------------------------------------'' '
                                        + 'UNION ALL '
                                        + 'SELECT ''-----| Execute sp_PivotExecute with parameters: columns and dimentions and main table name'' '
                                        + 'UNION ALL '
                                        + 'SELECT ''----------------------------------------------------------------------------------------------------'' '
                                        + 'UNION ALL '
                                        + 'SELECT ''EXEC [dbo].[sp_PivotExecute] @ColumnListWithActions, ' + '''''' + @TableName + '''''' + ';'''
                                        + 'UNION ALL '
                                        + 'SELECT ''----------------------------------------------------------------------------------------------------'' '                            
EXECUTE SP_EXECUTESQL @SQL_Code;

GO

CREATE PROCEDURE [dbo].[sp_PivotExecute]
(
@ColumnListWithActions  ColumnActionList ReadOnly
,@TableName                     nvarchar(128)
)
AS


--#######################################################################################################################
--###| Step 1 - Select our user-defined-table-variable into temp table
--#######################################################################################################################

IF OBJECT_ID('tempdb.dbo.#ColumnListWithActions', 'U') IS NOT NULL DROP TABLE #ColumnListWithActions; 
SELECT * INTO #ColumnListWithActions FROM @ColumnListWithActions;

--#######################################################################################################################
--###| Step 2 - Preparing lists of column groups as strings:
--#######################################################################################################################

DECLARE @ColumnName                     nvarchar(128)
DECLARE @Destiny                        nchar(1)

DECLARE @ListOfColumns_Stable           nvarchar(max)
DECLARE @ListOfColumns_Dimension    nvarchar(max)
DECLARE @ListOfColumns_Variable     nvarchar(max)
--############################
--###| Cursor for List of Stable Columns
--############################

DECLARE ColumnListStringCreator_S CURSOR FOR
SELECT      [ColumnName]
FROM        #ColumnListWithActions
WHERE       [Action] = 'S'
OPEN ColumnListStringCreator_S;
FETCH NEXT FROM ColumnListStringCreator_S
INTO @ColumnName
  WHILE @@FETCH_STATUS = 0

   BEGIN
        SELECT @ListOfColumns_Stable = ISNULL(@ListOfColumns_Stable, '') + ' [' + @ColumnName + '] ,';
        FETCH NEXT FROM ColumnListStringCreator_S INTO @ColumnName
   END

CLOSE ColumnListStringCreator_S;
DEALLOCATE ColumnListStringCreator_S;

--############################
--###| Cursor for List of Dimension Columns
--############################

DECLARE ColumnListStringCreator_D CURSOR FOR
SELECT      [ColumnName]
FROM        #ColumnListWithActions
WHERE       [Action] = 'D'
OPEN ColumnListStringCreator_D;
FETCH NEXT FROM ColumnListStringCreator_D
INTO @ColumnName
  WHILE @@FETCH_STATUS = 0

   BEGIN
        SELECT @ListOfColumns_Dimension = ISNULL(@ListOfColumns_Dimension, '') + ' [' + @ColumnName + '] ,';
        FETCH NEXT FROM ColumnListStringCreator_D INTO @ColumnName
   END

CLOSE ColumnListStringCreator_D;
DEALLOCATE ColumnListStringCreator_D;

--############################
--###| Cursor for List of Variable Columns
--############################

DECLARE ColumnListStringCreator_V CURSOR FOR
SELECT      [ColumnName]
FROM        #ColumnListWithActions
WHERE       [Action] = 'V'
OPEN ColumnListStringCreator_V;
FETCH NEXT FROM ColumnListStringCreator_V
INTO @ColumnName
  WHILE @@FETCH_STATUS = 0

   BEGIN
        SELECT @ListOfColumns_Variable = ISNULL(@ListOfColumns_Variable, '') + ' [' + @ColumnName + '] ,';
        FETCH NEXT FROM ColumnListStringCreator_V INTO @ColumnName
   END

CLOSE ColumnListStringCreator_V;
DEALLOCATE ColumnListStringCreator_V;

SELECT @ListOfColumns_Variable      = LEFT(@ListOfColumns_Variable, LEN(@ListOfColumns_Variable) - 1);
SELECT @ListOfColumns_Dimension = LEFT(@ListOfColumns_Dimension, LEN(@ListOfColumns_Dimension) - 1);
SELECT @ListOfColumns_Stable            = LEFT(@ListOfColumns_Stable, LEN(@ListOfColumns_Stable) - 1);

--#######################################################################################################################
--###| Step 3 - Preparing table with all possible connections between Dimension columns excluding NULLs
--#######################################################################################################################
DECLARE @DIM_TAB TABLE ([DIM_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO @DIM_TAB 
SELECT [DIM_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName] FROM #ColumnListWithActions WHERE [Action] = 'D';

DECLARE @DIM_ID smallint;
SELECT      @DIM_ID = 1;


DECLARE @SQL_Dimentions nvarchar(max);

IF OBJECT_ID('tempdb.dbo.##ALL_Dimentions', 'U') IS NOT NULL DROP TABLE ##ALL_Dimentions; 

SELECT @SQL_Dimentions      = 'SELECT [xxx_ID_xxx] = ROW_NUMBER() OVER (ORDER BY ' + @ListOfColumns_Dimension + '), ' + @ListOfColumns_Dimension
                                            + ' INTO ##ALL_Dimentions '
                                            + ' FROM (SELECT DISTINCT' + @ListOfColumns_Dimension + ' FROM  ' + @TableName
                                            + ' WHERE ' + (SELECT [ColumnName] FROM @DIM_TAB WHERE [DIM_ID] = @DIM_ID) + ' IS NOT NULL ';
                                            SELECT @DIM_ID = @DIM_ID + 1;
            WHILE @DIM_ID <= (SELECT MAX([DIM_ID]) FROM @DIM_TAB)
            BEGIN
            SELECT @SQL_Dimentions = @SQL_Dimentions + 'AND ' + (SELECT [ColumnName] FROM @DIM_TAB WHERE [DIM_ID] = @DIM_ID) +  ' IS NOT NULL ';
            SELECT @DIM_ID = @DIM_ID + 1;
            END

SELECT @SQL_Dimentions   = @SQL_Dimentions + ' )x';

EXECUTE SP_EXECUTESQL  @SQL_Dimentions;

--#######################################################################################################################
--###| Step 4 - Preparing table with all possible connections between Stable columns excluding NULLs
--#######################################################################################################################
DECLARE @StabPos_TAB TABLE ([StabPos_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO @StabPos_TAB 
SELECT [StabPos_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName] FROM #ColumnListWithActions WHERE [Action] = 'S';

DECLARE @StabPos_ID smallint;
SELECT      @StabPos_ID = 1;


DECLARE @SQL_MainStableColumnTable nvarchar(max);

IF OBJECT_ID('tempdb.dbo.##ALL_StableColumns', 'U') IS NOT NULL DROP TABLE ##ALL_StableColumns; 

SELECT @SQL_MainStableColumnTable       = 'SELECT xxx_ID_xxx = ROW_NUMBER() OVER (ORDER BY ' + @ListOfColumns_Stable + '), ' + @ListOfColumns_Stable
                                            + ' INTO ##ALL_StableColumns '
                                            + ' FROM (SELECT DISTINCT' + @ListOfColumns_Stable + ' FROM  ' + @TableName
                                            + ' WHERE ' + (SELECT [ColumnName] FROM @StabPos_TAB WHERE [StabPos_ID] = @StabPos_ID) + ' IS NOT NULL ';
                                            SELECT @StabPos_ID = @StabPos_ID + 1;
            WHILE @StabPos_ID <= (SELECT MAX([StabPos_ID]) FROM @StabPos_TAB)
            BEGIN
            SELECT @SQL_MainStableColumnTable = @SQL_MainStableColumnTable + 'AND ' + (SELECT [ColumnName] FROM @StabPos_TAB WHERE [StabPos_ID] = @StabPos_ID) +  ' IS NOT NULL ';
            SELECT @StabPos_ID = @StabPos_ID + 1;
            END

SELECT @SQL_MainStableColumnTable    = @SQL_MainStableColumnTable + ' )x';

EXECUTE SP_EXECUTESQL  @SQL_MainStableColumnTable;

--#######################################################################################################################
--###| Step 5 - Preparing table with all options ID
--#######################################################################################################################

DECLARE @FULL_SQL_1 NVARCHAR(MAX)
SELECT @FULL_SQL_1 = ''

DECLARE @i smallint

IF OBJECT_ID('tempdb.dbo.##FinalTab', 'U') IS NOT NULL DROP TABLE ##FinalTab; 

SELECT @FULL_SQL_1 = 'SELECT t.*, dim.[xxx_ID_xxx] '
                                    + ' INTO ##FinalTab '
                                    +   'FROM ' + @TableName + ' t '
                                    +   'JOIN ##ALL_Dimentions dim '
                                    +   'ON t.' + (SELECT [ColumnName] FROM @DIM_TAB WHERE [DIM_ID] = 1) + ' = dim.' + (SELECT [ColumnName] FROM @DIM_TAB WHERE [DIM_ID] = 1);
                                SELECT @i = 2                               
                                WHILE @i <= (SELECT MAX([DIM_ID]) FROM @DIM_TAB)
                                    BEGIN
                                    SELECT @FULL_SQL_1 = @FULL_SQL_1 + ' AND t.' + (SELECT [ColumnName] FROM @DIM_TAB WHERE [DIM_ID] = @i) + ' = dim.' + (SELECT [ColumnName] FROM @DIM_TAB WHERE [DIM_ID] = @i)
                                    SELECT @i = @i +1
                                END
EXECUTE SP_EXECUTESQL @FULL_SQL_1

--#######################################################################################################################
--###| Step 6 - Selecting final data
--#######################################################################################################################
DECLARE @STAB_TAB TABLE ([STAB_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO @STAB_TAB 
SELECT [STAB_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName]
FROM #ColumnListWithActions WHERE [Action] = 'S';

DECLARE @VAR_TAB TABLE ([VAR_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO @VAR_TAB 
SELECT [VAR_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName]
FROM #ColumnListWithActions WHERE [Action] = 'V';

DECLARE @y smallint;
DECLARE @x smallint;
DECLARE @z smallint;


DECLARE @FinalCode nvarchar(max)

SELECT @FinalCode = ' SELECT ID1.*'
                                        SELECT @y = 1
                                        WHILE @y <= (SELECT MAX([xxx_ID_xxx]) FROM ##FinalTab)
                                            BEGIN
                                                SELECT @z = 1
                                                WHILE @z <= (SELECT MAX([VAR_ID]) FROM @VAR_TAB)
                                                    BEGIN
                                                        SELECT @FinalCode = @FinalCode +    ', [ID' + CAST((@y) as varchar(10)) + '.' + (SELECT [ColumnName] FROM @VAR_TAB WHERE [VAR_ID] = @z) + '] =  ID' + CAST((@y + 1) as varchar(10)) + '.' + (SELECT [ColumnName] FROM @VAR_TAB WHERE [VAR_ID] = @z)
                                                        SELECT @z = @z + 1
                                                    END
                                                    SELECT @y = @y + 1
                                                END
        SELECT @FinalCode = @FinalCode + 
                                        ' FROM ( SELECT * FROM ##ALL_StableColumns)ID1';
                                        SELECT @y = 1
                                        WHILE @y <= (SELECT MAX([xxx_ID_xxx]) FROM ##FinalTab)
                                        BEGIN
                                            SELECT @x = 1
                                            SELECT @FinalCode = @FinalCode 
                                                                                + ' LEFT JOIN (SELECT ' +  @ListOfColumns_Stable + ' , ' + @ListOfColumns_Variable 
                                                                                + ' FROM ##FinalTab WHERE [xxx_ID_xxx] = ' 
                                                                                + CAST(@y as varchar(10)) + ' )ID' + CAST((@y + 1) as varchar(10))  
                                                                                + ' ON 1 = 1' 
                                                                                WHILE @x <= (SELECT MAX([STAB_ID]) FROM @STAB_TAB)
                                                                                BEGIN
                                                                                    SELECT @FinalCode = @FinalCode + ' AND ID1.' + (SELECT [ColumnName] FROM @STAB_TAB WHERE [STAB_ID] = @x) + ' = ID' + CAST((@y+1) as varchar(10)) + '.' + (SELECT [ColumnName] FROM @STAB_TAB WHERE [STAB_ID] = @x)
                                                                                    SELECT @x = @x +1
                                                                                END
                                            SELECT @y = @y + 1
                                        END

SELECT * FROM ##ALL_Dimentions;
EXECUTE SP_EXECUTESQL @FinalCode;

Ao executar a primeira consulta (passando o banco de dados de origem e o nome da tabela), você receberá uma consulta de execução pré-criada para o segundo SP, basta definir a coluna da sua fonte: + Estável + Valor (será usado para concentrar valores com base nisso) + Dim (coluna que você deseja usar para dinamizar)

Nomes e tipos de dados serão definidos automaticamente!

Não posso recomendá-lo para nenhum ambiente de produção, mas faz o trabalho para solicitações ad-hoc de BI.

Bartosz X
fonte
Tudo bem até Could not find stored procedure 'dbo.sp_PivotIt'.Algum conselho?
FAB
@DarXyde sory Eu devo misturar 2 versões, recompile e tente novamente #
Bartosz X
Obrigado Bartosz, consegui usar algumas das idéias do seu script e fiz o que já tinha em mente, mas mesmo assim, obrigado por atualizá-lo :). Eu deveria ter pensado em mudar essa linha, mas honestamente, pensei que é um procedimento armazenado que você esqueceu que não é o padrão no sistema ou algo assim. Vou executá-lo quando me aproximar desse projeto novamente e atualizar aqui!
FAB
11
Ainda é o nome errado do SP, mas agora que sei o que há de errado, fácil de corrigir: sp_Pivot_Executemude para proc_PivotExecute.
FAB
@DarXyde - graças, para detectar que - isso já foi corrigido
Bartosz X