Tuesday, March 20, 2012

Bulk Insert Problem

I receive near about 50 text files every month each has near about 1lakh to
3
lakh rows. they come in various formats (few with , delimiters, few with
field qualifiers, few with mix and match of both, and few files with fixed
format string)
so i use bulk insert to copy them into sql server using format files.
Everything works fine so far.
the problem is, in those txt files some times few lines can be either too
long or too short. in that case bulk insert totally rejects the entire file.
That is i dont want the process to stop just because few lines were corrupt.
is there anyway through which i can insert the data by skipping those few
corrupt lines.
Thanks for any help in advance.
This is urgentKitty,
Have a look at the -m and -e switches for bcp. This won't solve your problem
though, you need to fix the source data.
Mark Allison, SQL Server MVP
http://www.markallison.co.uk
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602m.html
"Kitty" wrote:

> I receive near about 50 text files every month each has near about 1lakh t
o 3
> lakh rows. they come in various formats (few with , delimiters, few with
> field qualifiers, few with mix and match of both, and few files with fixed
> format string)
> so i use bulk insert to copy them into sql server using format files.
> Everything works fine so far.
> the problem is, in those txt files some times few lines can be either too
> long or too short. in that case bulk insert totally rejects the entire fil
e.
> That is i dont want the process to stop just because few lines were corrup
t.
> is there anyway through which i can insert the data by skipping those few
> corrupt lines.
> Thanks for any help in advance.
> This is urgent|||Thanks for your time mark.
As you rightly said, this wont solve my problem. and i have no control over
source data, since it comes from various people from various countries all
over the world.
any more suggestions.
Thanks in advance
Kitty
"Mark Allison" wrote:
[vbcol=seagreen]
> Kitty,
> Have a look at the -m and -e switches for bcp. This won't solve your probl
em
> though, you need to fix the source data.
> --
> Mark Allison, SQL Server MVP
> http://www.markallison.co.uk
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602m.html
>
>
> "Kitty" wrote:
>sql

No comments:

Post a Comment