Showing posts with label source. Show all posts
Showing posts with label source. Show all posts

Tuesday, March 27, 2012

Bulk Load problem - error connecting

I get the following error when trying to use the SQL Bulk Load object.

"Error connecting to the data source." The datasource is used right above this code to successfully read from the database. It is the last line that blows up.

Here is the problem code:

connStr = "provider=SQLNCLI;Data Source=myserver;Initial Catalog=mydb;Integrated Security=True"

Dim objBL As New SQLXMLBULKLOADLib.SQLXMLBulkLoad

objBL.ConnectionString = connStr

objBL.BulkLoad = True

objBL.XMLFragment = True

objBL.KeepIdentity = False

objBL.ErrorLogFile = "C:\BulkLoadErrors.xml"

objBL.Execute(SchemaFile, datafile)

(SchemaFile and datafile are strings containing the full file name and path)

Use the connection string as follows :

ConnStr = "provider=sqloledb;data source=myserver;database=mydb;integrated security=SSPI;"

It should work.

Thanks.

Naras.

|||Thank you! That did it. Annoying part is the connection string I was using was taken straight from the MSDN examples.

Bulk load from Access to SQL 2003

I am trying to configure a bulk table upload from a 2003 Access table to a matching table in SQL with SSIS. I can configure the source file but am unable to configure the destination. When I configure the SQL source and use SQL Native Client I get an error message of:

The selected connection manager uses an earlier version of a SQL server provider. Bulk insert requires a connection that uses a SQL server 2005 provider.

When I go through the new connection setup I don't see any available provider named like that. I believe the SQL server I am loading to is a 2003 version.

try using the ole db destination. it uses bulk loading by design.|||

Thanks, that seemed to work to pull the table up. Pushing takes forever.

Sunday, March 25, 2012

Bulk Insert With Identity Field

Hi All,
Can I Bulk Insert to to SQL Table with a Identity Column in it?
My Source is a text file with 17 Columns and My Target is a SQL Server table
with 18 Columns (all 17 column of the source + 1 Identity Column as Primary
KEY).
So In this Situation How can i Bulk Insert to the SQL Table from the text
file. Please give small example if possible.
Also If my SQL table is have 2 More Extra Column Can I Boul Insert from the
above source?
Ex: Total 19 Columns ( all 17 columns of the source text file + 1 Identity
Column + 1 Extra column). If I want to Insert into the 17 columns and I want
the Indetity column to generate auto numbers and the Last Extra Column to be
Filled with Some "Char (1)" Value. Is that Possible?
Thanks for any Help or suggestions
Prabhat
using bulk insert there is a keepidentity parameter, using bcp it is -E...
Both are documented in books on line
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"Prabhat" <not_a_mail@.hotmail.com> wrote in message
news:OKa8j8unEHA.3868@.TK2MSFTNGP11.phx.gbl...
> Hi All,
> Can I Bulk Insert to to SQL Table with a Identity Column in it?
> My Source is a text file with 17 Columns and My Target is a SQL Server
table
> with 18 Columns (all 17 column of the source + 1 Identity Column as
Primary
> KEY).
> So In this Situation How can i Bulk Insert to the SQL Table from the text
> file. Please give small example if possible.
> Also If my SQL table is have 2 More Extra Column Can I Boul Insert from
the
> above source?
> Ex: Total 19 Columns ( all 17 columns of the source text file + 1 Identity
> Column + 1 Extra column). If I want to Insert into the 17 columns and I
want
> the Indetity column to generate auto numbers and the Last Extra Column to
be
> Filled with Some "Char (1)" Value. Is that Possible?
> Thanks for any Help or suggestions
> Prabhat
>
|||Thanks for the Hint. I have seen that in BOL but did not get any Example.
Can you suggest any site or give me a small Example where the Target table
has a Identity Field but the Source does not have the value for Identity
Column.
Thanks
Prabhat
"Wayne Snyder" <wayne.nospam.snyder@.mariner-usa.com> wrote in message
news:#m#6mmwnEHA.3900@.TK2MSFTNGP10.phx.gbl...[vbcol=seagreen]
> using bulk insert there is a keepidentity parameter, using bcp it is -E...
> Both are documented in books on line
> --
> Wayne Snyder, MCDBA, SQL Server MVP
> Mariner, Charlotte, NC
> www.mariner-usa.com
> (Please respond only to the newsgroups.)
> I support the Professional Association of SQL Server (PASS) and it's
> community of SQL Server professionals.
> www.sqlpass.org
> "Prabhat" <not_a_mail@.hotmail.com> wrote in message
> news:OKa8j8unEHA.3868@.TK2MSFTNGP11.phx.gbl...
> table
> Primary
text[vbcol=seagreen]
> the
Identity[vbcol=seagreen]
> want
to
> be
>

Bulk Insert With Identity Field

Hi All,
Can I Bulk Insert to to SQL Table with a Identity Column in it?
My Source is a text file with 17 Columns and My Target is a SQL Server table
with 18 Columns (all 17 column of the source + 1 Identity Column as Primary
KEY).
So In this Situation How can i Bulk Insert to the SQL Table from the text
file. Please give small example if possible.
Also If my SQL table is have 2 More Extra Column Can I Boul Insert from the
above source?
Ex: Total 19 Columns ( all 17 columns of the source text file + 1 Identity
Column + 1 Extra column). If I want to Insert into the 17 columns and I want
the Indetity column to generate auto numbers and the Last Extra Column to be
Filled with Some "Char (1)" Value. Is that Possible?
Thanks for any Help or suggestions
Prabhatusing bulk insert there is a keepidentity parameter, using bcp it is -E...
Both are documented in books on line
--
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"Prabhat" <not_a_mail@.hotmail.com> wrote in message
news:OKa8j8unEHA.3868@.TK2MSFTNGP11.phx.gbl...
> Hi All,
> Can I Bulk Insert to to SQL Table with a Identity Column in it?
> My Source is a text file with 17 Columns and My Target is a SQL Server
table
> with 18 Columns (all 17 column of the source + 1 Identity Column as
Primary
> KEY).
> So In this Situation How can i Bulk Insert to the SQL Table from the text
> file. Please give small example if possible.
> Also If my SQL table is have 2 More Extra Column Can I Boul Insert from
the
> above source?
> Ex: Total 19 Columns ( all 17 columns of the source text file + 1 Identity
> Column + 1 Extra column). If I want to Insert into the 17 columns and I
want
> the Indetity column to generate auto numbers and the Last Extra Column to
be
> Filled with Some "Char (1)" Value. Is that Possible?
> Thanks for any Help or suggestions
> Prabhat
>|||Thanks for the Hint. I have seen that in BOL but did not get any Example.
Can you suggest any site or give me a small Example where the Target table
has a Identity Field but the Source does not have the value for Identity
Column.
Thanks
Prabhat
"Wayne Snyder" <wayne.nospam.snyder@.mariner-usa.com> wrote in message
news:#m#6mmwnEHA.3900@.TK2MSFTNGP10.phx.gbl...
> using bulk insert there is a keepidentity parameter, using bcp it is -E...
> Both are documented in books on line
> --
> Wayne Snyder, MCDBA, SQL Server MVP
> Mariner, Charlotte, NC
> www.mariner-usa.com
> (Please respond only to the newsgroups.)
> I support the Professional Association of SQL Server (PASS) and it's
> community of SQL Server professionals.
> www.sqlpass.org
> "Prabhat" <not_a_mail@.hotmail.com> wrote in message
> news:OKa8j8unEHA.3868@.TK2MSFTNGP11.phx.gbl...
> > Hi All,
> >
> > Can I Bulk Insert to to SQL Table with a Identity Column in it?
> >
> > My Source is a text file with 17 Columns and My Target is a SQL Server
> table
> > with 18 Columns (all 17 column of the source + 1 Identity Column as
> Primary
> > KEY).
> >
> > So In this Situation How can i Bulk Insert to the SQL Table from the
text
> > file. Please give small example if possible.
> >
> > Also If my SQL table is have 2 More Extra Column Can I Boul Insert from
> the
> > above source?
> > Ex: Total 19 Columns ( all 17 columns of the source text file + 1
Identity
> > Column + 1 Extra column). If I want to Insert into the 17 columns and I
> want
> > the Indetity column to generate auto numbers and the Last Extra Column
to
> be
> > Filled with Some "Char (1)" Value. Is that Possible?
> >
> > Thanks for any Help or suggestions
> > Prabhat
> >
> >
>sql

Bulk Insert vs. Data Flow Task (different row results using flat file source)

I'm importing a large csv file two different ways - one with Bulk Import Task and the other way with the Data Flow Task (flat file source -> OLE DB destination).

With the Bulk Import Task I'm putting all the csv rows in one column. With the Data Flow Task I'm mapping each csv value to it's own column in the SQL table.

I used two different flat file sources and got the following:

Flat file 1: Bulk Import Task = 12,649,499 rows; Data Flow Task = 4,215,817 rows
Flat file 2: Bulk Import Task = 3,403,254 rows; Data Flow Task = 1,134,359 rows

Anyone have any guess as to why this is happening?

It seems that there's a factor of 3 in between the two components, which could mean that Flat File Connection Manager is interpereting 3 rows as 1. This might be related with the row delimiter in the Flat File connection manager.

Can you try to use preview in the Flat File connection manager, and see if rows show up correctly?

|||Yeah, thanks for the reply. I've checked the preview and it looks fine. I've also tracked down some of the skipped rows and there's nothing signficant about them. They look exactly the same as all the others. Perhaps there is a hidden character or something? Is this possible?|||

Can you compare the rows imported by FlatFile with the rows imported by Bulk insert task and see if there is any difference between them?

Thanks,
Ovidiu Burlacu

|||yes, i can do that and the resolution is that they look exactly the same. ug. there really is no indication i can see to why a rows get dropped. they are not the same type, they don't have rows before them or after them with any significant difference. it's really very puzzling.|||

Can you put a data viewer in your pipeline and see what rows pass through at execution time?

Thanks,
Ovidiu Burlacu

Wednesday, March 7, 2012

Bulk insert and vb.net - URGENT

Hello.
I want to use bulk insert in order to import a huge file, lets say
"Source.txt" in my db, lets say "MyDB".
The data in "Source.txt" are numerical and date format. Values are seperated
with the tab character.
When I use :
BULK INSERT [owner].[dbname].[tablename] FROM
'c:\Source.txt' WITH (ROWTERMINATOR = '\n', FIELDTERMINATOR = '\t' ,
KEEPNULLS )
everything works fine.
Now, I want to add a datetime field in the beginning of each line in
source.txt.
I do this by :
<...open Source.txt as 1...>
<...open Altered.txt as 2...>
OldLine=LineInput(1)
NewLine="...DateVariable..." & vbTab &
OldLine
Print 2, OldLine & vbNewLine
<...close files...>
Why does
BULK INSERT [owner].[dbname].[tablename] FROM
'c:\Altered.txt' WITH (ROWTERMINATOR = '\n', FIELDTERMINATOR = '\t' ,
KEEPNULLS )
doesn't work?
The error message I get is
Bulk insert data conversion error (type mismatch) for row 1,
column 385 (...column description...).
I have tried with different row terminators but no success.
I suspect that while creating the Altered.txt, 2 row terminator characters
are added instead of 1.
Please any help will be much appreciated.
Thank you,
VasilisDid you also recreate the table with the datetime as the first column? File
fields are mapped to target table columns by ordinal position unless you
specify a format file.
Hope this helps.
Dan Guzman
SQL Server MVP
"Vasilis X" <v.hantziaras@.interwind.gr> wrote in message
news:du1d59$2pa3$1@.ulysses.noc.ntua.gr...
> Hello.
> I want to use bulk insert in order to import a huge file, lets say
> "Source.txt" in my db, lets say "MyDB".
> The data in "Source.txt" are numerical and date format. Values are
> seperated with the tab character.
> When I use :
> BULK INSERT [owner].[dbname].[tablename] FROM
> 'c:\Source.txt' WITH (ROWTERMINATOR = '\n', FIELDTERMINATOR = '\t' ,
> KEEPNULLS )
> everything works fine.
> Now, I want to add a datetime field in the beginning of each line in
> source.txt.
> I do this by :
> <...open Source.txt as 1...>
> <...open Altered.txt as 2...>
> OldLine=LineInput(1)
> NewLine="...DateVariable..." & vbTab & OldLine
> Print 2, OldLine & vbNewLine
> <...close files...>
>
> Why does
> BULK INSERT [owner].[dbname].[tablename] FROM
> 'c:\Altered.txt' WITH (ROWTERMINATOR = '\n', FIELDTERMINATOR = '\t' ,
> KEEPNULLS )
> doesn't work?
> The error message I get is
> Bulk insert data conversion error (type mismatch) for row
> 1, column 385 (...column description...).
>
> I have tried with different row terminators but no success.
> I suspect that while creating the Altered.txt, 2 row terminator characters
> are added instead of 1.
> Please any help will be much appreciated.
> Thank you,
> Vasilis
>
>
>|||Yes. The table stracture was changed accordingly.
"Dan Guzman" <guzmanda@.nospam-online.sbcglobal.net> wrote in message
news:%23WQyROGPGHA.2124@.TK2MSFTNGP14.phx.gbl...
> Did you also recreate the table with the datetime as the first column?
> File fields are mapped to target table columns by ordinal position unless
> you specify a format file.
>
> --
> Hope this helps.
> Dan Guzman
> SQL Server MVP
> "Vasilis X" <v.hantziaras@.interwind.gr> wrote in message
> news:du1d59$2pa3$1@.ulysses.noc.ntua.gr...
>
>|||>>> Print 2, OldLine & vbNewLine
Reviewing your pseudo-code, it looks to me like your are writing out the
original record instead of the new one with the date. Check the output file
to ensure it is as expected. If you still have problems, post the actual
code.
Hope this helps.
Dan Guzman
SQL Server MVP
"Vasilis X" <v.hantziaras@.interwind.gr> wrote in message
news:du1gjs$14o$1@.ulysses.noc.ntua.gr...
> Yes. The table stracture was changed accordingly.
>
> "Dan Guzman" <guzmanda@.nospam-online.sbcglobal.net> wrote in message
> news:%23WQyROGPGHA.2124@.TK2MSFTNGP14.phx.gbl...
>
>|||Yes, you are right.
I actually use Print 2, Newline & vbNewLine.
Following is the actual code :
FileOpen(2, Bulk_File, OpenMode.Output)
While Not EOF(1)
Input(1, lineText)
'create new line
Dim NewLine As String
NewLine = NewDateVariable & vbTab & lineText
'print new line
Print(2, NewLine & vbCrLf) 'Also tried with NewLine, NewLine &
vbNewLine, even with a simple terminator, eg "!" and changed bulk command
syntax to ...ROWTERMINATOR='!'...
End While
FileClose()
Dim bulkCom As String
bulkCom = "BULK INSERT [mydatabase].[owner].[mytable] FROM '" & Bulk_File &
"' WITH ( FIELDTERMINATOR = '\t' , ROWTERMINATOR = '\n'' , KEEPNULLS )"
bulkCom &= WED.ExecuteCommand(bulkCom, False)
End If
WED is a class to communicate with the server. It doesn't add anything to
the command that is to be executed
Bulk_File is the file to be imported|||Did you view the output file to make sure the format is as expected?
Unfortunately, this code snippet doesn't include the NewDataVariable
assignment, which may be part of the problem. I'd like to have enough code
so that I can reproduce your problem in my environment.
I notice that you use NewLine as a variable name. Perhaps
System.Environment.NewLine is being used instead so the output is not as
expected.
Hope this helps.
Dan Guzman
SQL Server MVP
"Vasilis X" <v.hantziaras@.interwind.gr> wrote in message
news:du1kki$c2q$1@.ulysses.noc.ntua.gr...
> Yes, you are right.
> I actually use Print 2, Newline & vbNewLine.
> Following is the actual code :
> FileOpen(2, Bulk_File, OpenMode.Output)
> While Not EOF(1)
> Input(1, lineText)
> 'create new line
> Dim NewLine As String
> NewLine = NewDateVariable & vbTab & lineText
> 'print new line
> Print(2, NewLine & vbCrLf) 'Also tried with NewLine, NewLine
> & vbNewLine, even with a simple terminator, eg "!" and changed bulk
> command syntax to ...ROWTERMINATOR='!'...
> End While
> FileClose()
> Dim bulkCom As String
> bulkCom = "BULK INSERT [mydatabase].[owner].[mytable] FROM '" & Bulk_File
> & "' WITH ( FIELDTERMINATOR = '\t' , ROWTERMINATOR = '\n'' , KEEPNULLS )"
> bulkCom &= WED.ExecuteCommand(bulkCom, False)
> End If
>
> WED is a class to communicate with the server. It doesn't add anything to
> the command that is to be executed
> Bulk_File is the file to be imported
>
>|||The first thing I'd check is that the file you are getting has the correct
separator and terminator bytes. Possibly Print is adding its own line
terminators, e.g., vbCrLf. The easiest way to check is to open a Command
window, enter debug source.txt, and then type d at the - prompt. That will
dump the start of the file in ASCII and Hex. Enter d again to dump the next
128 bytes. Enter q to quit out of debug.
Your lines should be terminated by 0A. If they are terminated by 0D 0A, for
example, VB is adding a CR and LF (the standard Windows text file line
terminator), and you will have to modify your program or the BULK INSERT
accordingly.
HTH,
Mike Abraham
"Vasilis X" <v.hantziaras@.interwind.gr> wrote in message
news:du1kki$c2q$1@.ulysses.noc.ntua.gr...
> Yes, you are right.
> I actually use Print 2, Newline & vbNewLine.
> Following is the actual code :
> FileOpen(2, Bulk_File, OpenMode.Output)
> While Not EOF(1)
> Input(1, lineText)
> 'create new line
> Dim NewLine As String
> NewLine = NewDateVariable & vbTab & lineText
> 'print new line
> Print(2, NewLine & vbCrLf) 'Also tried with NewLine, NewLine
> & vbNewLine, even with a simple terminator, eg "!" and changed bulk
> command syntax to ...ROWTERMINATOR='!'...
> End While
> FileClose()
> Dim bulkCom As String
> bulkCom = "BULK INSERT [mydatabase].[owner].[mytable] FROM '" & Bulk_File
> & "' WITH ( FIELDTERMINATOR = '\t' , ROWTERMINATOR = '\n'' , KEEPNULLS )"
> bulkCom &= WED.ExecuteCommand(bulkCom, False)
> End If
>
> WED is a class to communicate with the server. It doesn't add anything to
> the command that is to be executed
> Bulk_File is the file to be imported
>
>

Sunday, February 19, 2012

Bulk copy XML data to SQL data source

Can anyone provide an expample of bulk copying XML data to a SQL table. I am also looking at using column mapping so that I can map fields and also insert a new GUID into the key of the SQL table.

Many thanks

There are some ways, take a look at this link:

Importing Bulk Data by Using BULK INSERT or OPENROWSET(BULK...): http://msdn2.microsoft.com/en-us/library/ms175915.aspx

And examples for bulking XML data can be found here:

http://msdn2.microsoft.com/en-us/library/ms191184.aspx

Friday, February 10, 2012

Build Report Model from Analysis Service Database

I try to build a report model that have data source point to AS Database, but
i can't.
This is what i try: change the provider to OLE Db AS 9.0 and select the AS
database, but the data source view can't detect a data source.
another way i try is using an existing AS project, but the project combo box
is empty.
Pls let me know way to do this. Thank you.Just to add-on: I am using Microsoft SQL Server 2005 CTP version
"Chai Ling" wrote:
> I try to build a report model that have data source point to AS Database, but
> i can't.
> This is what i try: change the provider to OLE Db AS 9.0 and select the AS
> database, but the data source view can't detect a data source.
> another way i try is using an existing AS project, but the project combo box
> is empty.
> Pls let me know way to do this. Thank you.

Build Query dialog for OLEDB Source Editor

Can i extend the "Query Builder" dialog of OLEDB Source Editor for developing custom source component?

No. None of the MS component/task UI 's can be inherited or extended, you will have to write your own.