BULK INSERT skips a row

Hi. Not sure if this is a bug with bulk insert or standard behaviour.
We have a flat file with a header record. A simplified example follows:
HEADER
John,Smith
Fred,Bloggs
But when we set bulk insert to skip the first row it actually skips the
John,Smith row as well. Run the script below after saving the CSV data out
to a file called user.csv...
create table [User]
(
FirstName varchar ( 32 ),
LastName varchar ( 32 )
)
bulk insert [User] from 'user.csv'
with
(
FIRSTROW = 2,
DATAFILETYPE = 'char',
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
select * from [user] -- only one record
Please help!!
McGy
[url]http://mcgy.blogspot.com[/url]Anyone with ideas on how to get around this limitation?
McGy
[url]http://mcgy.blogspot.com[/url]
"McGy" <anon@.anon.com> wrote in message
news:#1#7JPAVGHA.5332@.tk2msftngp13.phx.gbl...
> Hi. Not sure if this is a bug with bulk insert or standard behaviour.
> We have a flat file with a header record. A simplified example follows:
> HEADER
> John,Smith
> Fred,Bloggs
> But when we set bulk insert to skip the first row it actually skips the
> John,Smith row as well. Run the script below after saving the CSV data out
> to a file called user.csv...
> create table [User]
> (
> FirstName varchar ( 32 ),
> LastName varchar ( 32 )
> )
> bulk insert [User] from 'user.csv'
> with
> (
> FIRSTROW = 2,
> DATAFILETYPE = 'char',
> FIELDTERMINATOR = ',',
> ROWTERMINATOR = '\n'
> )
> select * from [user] -- only one record
> Please help!!
> --
> McGy
> [url]http://mcgy.blogspot.com[/url]
>
>|||Some solutions are
1. Pre-process the file to remove the header line (use a DOS
or Unix-like utility, for example).
2. Bulk insert the data into a staging table with one column,
then parse the data into columns using SQL instead of the
bulk insert process.
3. Use SQL Server Integration Services. I'm no expert, but I believe
it can skip (or import into a different table) rows based on the
contents of the row, which bulk insert cannot do.
The FIRSTROW parameter for bulk insert, as you have discovered,
is not a FIRST_LINE_OF_THE_TEXT_FILE parameter.
Steve Kass
Drew University
McGy wrote:

>Anyone with ideas on how to get around this limitation?
>--
>McGy
>[url]http://mcgy.blogspot.com[/url]
>
>"McGy" <anon@.anon.com> wrote in message
>news:#1#7JPAVGHA.5332@.tk2msftngp13.phx.gbl...
>
>
>|||McGy (anon@.anon.com) writes:
Alas, this is standard behaviour.
BULK INSERT and BCP live in a very squared world. They don't think in
terms of lines, they think in terms of records and fields. (But calls them
rows and columns to matters.)
The first field is from the beginning of file until the terminator for
the first field. (For terminated fields. Fields can also be fix-length,
or include a length-prefix.) The second field lasts until the terminator
for that field. And so on, up to the last field which lasts until the
terminator for that field. That terminator is called a "row terminator",
but as long as BCP is not reading the last field, the terminator carries
no meaning. Once the last field has been read, bulk copy starts with
the first field again.
This has some advantages and divantages. On the flip side, is that
you can handle embedded row terminators, for instance embedded newlines
(except in the last field). The bad part is that you can't skip a header
which does not conform with the file. Because as far as BCP is concerned,
that header is part of the first field.
The workaround is to run a program first that strips the header, which
for a large file is not very appealing.
I know I have subumitted a suggestion for an improvement to bulk copy
for a future release of SQL Server, so that you should be able to describe
a header to be skipped. The suggestion is on
http://lab.msdn.microsoft.com/productfeedback/ but alas, the site
appears to experience some problems right now, so I cannot give the
exact link. But when it's back, search for by name, "bulk" and SQL 2005
to find it and vote for it.
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx

Building a report simply by passing a Recordset.

I just attented a Microsoft event where I ask Bernard Wong of Microsoft, if
it was possible to pass a recordset (and some header information) via code
to produce a simple report. He indicated of course and it is well
documented. Does anybody know how to perform this operation, or know where
the documentation on this type of process is?
Byron...The only way I know of how to do this is to read the RDL specification
(posted on Microsoft website). Parse the recordset and make decisions of
things like format, column size and heading. Create the RDL (which is an XML
file but that doesn't mean you don't have to understand the spec). Then you
would have to deploy it with a unique name and then have the client use the
unique name. If using the new controls you could instead use them in local
mode, pass the rdlc and the dataset to the control and have it render it.
The specification is well documented. There are some people who are doing
this. In my mind it is definitely non-trivial.
Bruce Loehle-Conger
MVP SQL Server Reporting Services
"Byron Hopp" <bhopp@.matrixcomputer.com> wrote in message
news:uKm0uuS7FHA.2012@.TK2MSFTNGP14.phx.gbl...
>I just attented a Microsoft event where I ask Bernard Wong of Microsoft, if
>it was possible to pass a recordset (and some header information) via code
>to produce a simple report. He indicated of course and it is well
>documented. Does anybody know how to perform this operation, or know where
>the documentation on this type of process is?
> Byron...
>|||With RS 2005, the answer to your question is depends on how the report is to
be rendered. If you use the Report Viewer controls in local mode, you can
bind the dataset to the report. If you need to pass the dataset to a
server-based report, you need a custom data extension
(http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=B8468707-56EF-4864-AC51-D83FC3273FE5).
--
HTH,
---
Teo Lachev, MVP, MCSD, MCT
"Microsoft Reporting Services in Action"
"Applied Microsoft Analysis Services 2005"
Home page and blog: http://www.prologika.com/
---
"Byron Hopp" <bhopp@.matrixcomputer.com> wrote in message
news:uKm0uuS7FHA.2012@.TK2MSFTNGP14.phx.gbl...
>I just attented a Microsoft event where I ask Bernard Wong of Microsoft, if
>it was possible to pass a recordset (and some header information) via code
>to produce a simple report. He indicated of course and it is well
>documented. Does anybody know how to perform this operation, or know where
>the documentation on this type of process is?
> Byron...
>|||I've been trying to do that (RS2005) without much success. I have a rdl file
I have brought in as a rdlc file. I would like to set the datasource to
something in code. If you have any info to point me in the right direction I
would appreciate it. All the examples I see all deal with doing everything
from the designer.
Bruce Loehle-Conger
MVP SQL Server Reporting Services
"Teo Lachev [MVP]" <teo.lachev@.nospam.prologika.com> wrote in message
news:%23K%23xL8s7FHA.2600@.tk2msftngp13.phx.gbl...
> With RS 2005, the answer to your question is depends on how the report is
> to be rendered. If you use the Report Viewer controls in local mode, you
> can bind the dataset to the report. If you need to pass the dataset to a
> server-based report, you need a custom data extension
> (http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=B8468707-56EF-4864-AC51-D83FC3273FE5).
> --
> HTH,
> ---
> Teo Lachev, MVP, MCSD, MCT
> "Microsoft Reporting Services in Action"
> "Applied Microsoft Analysis Services 2005"
> Home page and blog: http://www.prologika.com/
> ---
> "Byron Hopp" <bhopp@.matrixcomputer.com> wrote in message
> news:uKm0uuS7FHA.2012@.TK2MSFTNGP14.phx.gbl...
>>I just attented a Microsoft event where I ask Bernard Wong of Microsoft,
>>if it was possible to pass a recordset (and some header information) via
>>code to produce a simple report. He indicated of course and it is well
>>documented. Does anybody know how to perform this operation, or know
>>where the documentation on this type of process is?
>> Byron...
>>
>|||Nevermind, I got it.
--
Bruce Loehle-Conger
MVP SQL Server Reporting Services
"Bruce L-C [MVP]" <bruce_lcNOSPAM@.hotmail.com> wrote in message
news:%2393%23RAx7FHA.2092@.TK2MSFTNGP12.phx.gbl...
> I've been trying to do that (RS2005) without much success. I have a rdl
> file I have brought in as a rdlc file. I would like to set the datasource
> to something in code. If you have any info to point me in the right
> direction I would appreciate it. All the examples I see all deal with
> doing everything from the designer.
>
> --
> Bruce Loehle-Conger
> MVP SQL Server Reporting Services
> "Teo Lachev [MVP]" <teo.lachev@.nospam.prologika.com> wrote in message
> news:%23K%23xL8s7FHA.2600@.tk2msftngp13.phx.gbl...
>> With RS 2005, the answer to your question is depends on how the report is
>> to be rendered. If you use the Report Viewer controls in local mode, you
>> can bind the dataset to the report. If you need to pass the dataset to a
>> server-based report, you need a custom data extension
>> (http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=B8468707-56EF-4864-AC51-D83FC3273FE5).
>> --
>> HTH,
>> ---
>> Teo Lachev, MVP, MCSD, MCT
>> "Microsoft Reporting Services in Action"
>> "Applied Microsoft Analysis Services 2005"
>> Home page and blog: http://www.prologika.com/
>> ---
>> "Byron Hopp" <bhopp@.matrixcomputer.com> wrote in message
>> news:uKm0uuS7FHA.2012@.TK2MSFTNGP14.phx.gbl...
>>I just attented a Microsoft event where I ask Bernard Wong of Microsoft,
>>if it was possible to pass a recordset (and some header information) via
>>code to produce a simple report. He indicated of course and it is well
>>documented. Does anybody know how to perform this operation, or know
>>where the documentation on this type of process is?
>> Byron...
>>
>>
>|||Bruce,
Would you mind sharing your success?
Thanks,
Byron...
"Bruce L-C [MVP]" <bruce_lcNOSPAM@.hotmail.com> wrote in message
news:uvi2AQ%237FHA.2716@.TK2MSFTNGP11.phx.gbl...
> Nevermind, I got it.
> --
> Bruce Loehle-Conger
> MVP SQL Server Reporting Services
> "Bruce L-C [MVP]" <bruce_lcNOSPAM@.hotmail.com> wrote in message
> news:%2393%23RAx7FHA.2092@.TK2MSFTNGP12.phx.gbl...
>> I've been trying to do that (RS2005) without much success. I have a rdl
>> file I have brought in as a rdlc file. I would like to set the datasource
>> to something in code. If you have any info to point me in the right
>> direction I would appreciate it. All the examples I see all deal with
>> doing everything from the designer.
>>
>> --
>> Bruce Loehle-Conger
>> MVP SQL Server Reporting Services
>> "Teo Lachev [MVP]" <teo.lachev@.nospam.prologika.com> wrote in message
>> news:%23K%23xL8s7FHA.2600@.tk2msftngp13.phx.gbl...
>> With RS 2005, the answer to your question is depends on how the report
>> is to be rendered. If you use the Report Viewer controls in local mode,
>> you can bind the dataset to the report. If you need to pass the dataset
>> to a server-based report, you need a custom data extension
>> (http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=B8468707-56EF-4864-AC51-D83FC3273FE5).
>> --
>> HTH,
>> ---
>> Teo Lachev, MVP, MCSD, MCT
>> "Microsoft Reporting Services in Action"
>> "Applied Microsoft Analysis Services 2005"
>> Home page and blog: http://www.prologika.com/
>> ---
>> "Byron Hopp" <bhopp@.matrixcomputer.com> wrote in message
>> news:uKm0uuS7FHA.2012@.TK2MSFTNGP14.phx.gbl...
>>I just attented a Microsoft event where I ask Bernard Wong of Microsoft,
>>if it was possible to pass a recordset (and some header information) via
>>code to produce a simple report. He indicated of course and it is well
>>documented. Does anybody know how to perform this operation, or know
>>where the documentation on this type of process is?
>> Byron...
>>
>>
>>
>|||You get a working demo by downloading the chapter 18 source code of my book
(http://www.prologika.com/Books/0976635305/code/).
--
HTH,
---
Teo Lachev, MVP, MCSD, MCT
"Microsoft Reporting Services in Action"
"Applied Microsoft Analysis Services 2005"
Home page and blog: http://www.prologika.com/
---
"Byron Hopp" <bhopp@.matrixcomputer.com> wrote in message
news:uUaPKE$8FHA.3416@.TK2MSFTNGP15.phx.gbl...
> Bruce,
> Would you mind sharing your success?
> Thanks,
> Byron...
> "Bruce L-C [MVP]" <bruce_lcNOSPAM@.hotmail.com> wrote in message
> news:uvi2AQ%237FHA.2716@.TK2MSFTNGP11.phx.gbl...
>> Nevermind, I got it.
>> --
>> Bruce Loehle-Conger
>> MVP SQL Server Reporting Services
>> "Bruce L-C [MVP]" <bruce_lcNOSPAM@.hotmail.com> wrote in message
>> news:%2393%23RAx7FHA.2092@.TK2MSFTNGP12.phx.gbl...
>> I've been trying to do that (RS2005) without much success. I have a rdl
>> file I have brought in as a rdlc file. I would like to set the
>> datasource to something in code. If you have any info to point me in the
>> right direction I would appreciate it. All the examples I see all deal
>> with doing everything from the designer.
>>
>> --
>> Bruce Loehle-Conger
>> MVP SQL Server Reporting Services
>> "Teo Lachev [MVP]" <teo.lachev@.nospam.prologika.com> wrote in message
>> news:%23K%23xL8s7FHA.2600@.tk2msftngp13.phx.gbl...
>> With RS 2005, the answer to your question is depends on how the report
>> is to be rendered. If you use the Report Viewer controls in local mode,
>> you can bind the dataset to the report. If you need to pass the dataset
>> to a server-based report, you need a custom data extension
>> (http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=B8468707-56EF-4864-AC51-D83FC3273FE5).
>> --
>> HTH,
>> ---
>> Teo Lachev, MVP, MCSD, MCT
>> "Microsoft Reporting Services in Action"
>> "Applied Microsoft Analysis Services 2005"
>> Home page and blog: http://www.prologika.com/
>> ---
>> "Byron Hopp" <bhopp@.matrixcomputer.com> wrote in message
>> news:uKm0uuS7FHA.2012@.TK2MSFTNGP14.phx.gbl...
>>I just attented a Microsoft event where I ask Bernard Wong of
>>Microsoft, if it was possible to pass a recordset (and some header
>>information) via code to produce a simple report. He indicated of
>>course and it is well documented. Does anybody know how to perform
>>this operation, or know where the documentation on this type of process
>>is?
>> Byron...
>>
>>
>>
>>
>