Tuesday, July 25, 2006
How big is your CSV?
I downloaded the Woolworths data feed from Affiliate Window the other day.
Yikes!! That's a CSV file with over 200,000 rows.
Now normally I'd want to open up a CSV and make sure it's all in order, and maybe filter out stuff I don't want. So there I am, double clicking on this CSV, starting Excel when - eek! - it hits Excel's limit on the number of rows, 65536.
So what do you do with a CSV file of over 200,000 rows? I don't fancy putting it straight onto my production SQL database as I only want a subset of the data and it seems a bit OTT.
I guess I will have to try and import it into Access and do what I need to do in that.
Got me thinking.... I wonder who wins the prize for the biggest data feed in all of affiliate marketing? I know size isn't everything but I reckon Woolies must be one of the front runners?
Yikes!! That's a CSV file with over 200,000 rows.
Now normally I'd want to open up a CSV and make sure it's all in order, and maybe filter out stuff I don't want. So there I am, double clicking on this CSV, starting Excel when - eek! - it hits Excel's limit on the number of rows, 65536.
So what do you do with a CSV file of over 200,000 rows? I don't fancy putting it straight onto my production SQL database as I only want a subset of the data and it seems a bit OTT.
I guess I will have to try and import it into Access and do what I need to do in that.
Got me thinking.... I wonder who wins the prize for the biggest data feed in all of affiliate marketing? I know size isn't everything but I reckon Woolies must be one of the front runners?
Subscribe to Posts [Atom]









