lil-csv, a 1k JS module to parse and generate CSV files

I was struggling to find a small JavaScript module to parse CSV (Comma Separated Values) files. All the existing modules have one or more shortcomings:

  • Do not work in browsers;
  • Large (all I found are at least 10kb min.js.gz).
  • Cannot parse to deep objects.
  • Cannot generate CSV from deep objects.

I'm pretty sure CSV parsing can be implemented in less than 1Kb min.js.gz. So I did it.

Please meet the powerful lil-csv.

That's how large it is (v1.3.1):

1465 B: lil-csv.js.gz
       1313 B: lil-csv.js.br
       1315 B: lil-csv.modern.js.gz
       1201 B: lil-csv.modern.js.br
       1480 B: lil-csv.module.js.gz
       1327 B: lil-csv.module.js.br
       1524 B: lil-csv.umd.js.gz
       1359 B: lil-csv.umd.js.br

There are tradeoffs though.

  1. It doesn't accept streams but only the full CSV file content as a single string. (Also remember: Premature optimisation is the root of all evil.)
  2. ...you tell me...

Simple example

CSV to JavaScript objects

Let's say you have a CSV like this:

name,address,course
John Smith,"123 John St, CARLTON",Advanced Calculus
Any Newman,"4a/3a Church Ave, CROYDON",Advanced Calculus

Parse it to objects:

import { parse } from "lil-csv";
const objects = parse(fileContents);
console.log(objects);

/*
[
  {
    "name": "John Smith",
    "address": "123 John St, CARLTON",
    "course": "Advanced Calculus"
  },
  {
    "name": "Any Newman",
    "address": "4a/3a Church Ave, CROYDON",
    "course": "Advanced Calculus"
  }
]
*/

And stringify them back to CSV string:

import { generate } from "lil-csv";
const string = generate(objects);
console.log(string);

/*
name,address,course
John Smith,"123 John St, CARLTON",Advanced Calculus
Any Newman,"4a/3a Church Ave, CROYDON",Advanced Calculus
*/

So, in essence stringifying plus parsing is an idempotent operation:

assert.deepEqual(objects, parse(generate(objects)));

CSV to JavaScript array of arrays

If you just need arrays of strings (not objects) then here is how you do it:

const arrays = parse(fileContents, { header: false });
console.log(arrays);

/*
[
  ["name","address","course"],
  ["John Smith","123 John St, CARLTON","Advanced Calculus"],
  ["Any Newman","4a/3a Church Ave, CROYDON","Advanced Calculus"]
]
*/

Stringyfing back to CSV is simple:

const string = generate(arrays, { header: false });
console.log(string);

/*
name,address,course
John Smith,"123 John St, CARLTON",Advanced Calculus
Any Newman,"4a/3a Church Ave, CROYDON",Advanced Calculus
*/

Complex example

Parsing numbers, dates, booleans

In real world the data is rarely all strings. Often your objects have to have numbers, dates, booleans, etc. Here is how to parse CSV with all kind of data.

Let's parse some strings, dates, numbers and booleans from the following CSV file:

firstName,lastName,dob,price,completed
John,Smith,1999-01-15,123.00,Y
Alice,Dwarf,1991-11-24,123.00,N

Converting custom string to the JS objects, and leaving all other data as strings:

const people = parse(fileContents, {
  header: {
    "*": String,
    dob: v => v ? new Date(v) : null,
    price: v => isNaN(v) ? null : Number(v),
    completed: v => String(v).toUpperCase() === "Y",
  }
});

console.log(people);

/*
[
  {
    "firstName": "John",
    "lastName": "Smith",
    "dob": "1999-01-15T00:00:00.000Z",
    "price": 123.00,
    "completed": true
  },
  {
    "firstName": "Alice",
    "lastName": "Dwarf",
    "dob": "1991-11-24T00:00:00.000Z",
    "price": 123.00,
    "completed": false
  }
]
*/

Generating custom CSV

Here is how you can convert booleans to strings like "Y" and "N", and also convert JS Date to calendar dates like "YYYY-MM-DD", and add custom formatting to numbers like "123.00" instead of the default "123":

const string = generate(people, {
  header: {
    "*": String,
    dob: v => v ? new Date(v).toISOString().substr(0, 10) : "",
    price: v => isNaN(v) ? "" : Number(v).toFixed(2),
    completed: v => v ? "Y" : "N",
  }
});

console.log(string);

/*
firstName,lastName,dob,price,completed
John,Smith,1999-01-15,123.55,Y
Alice,Dwarf,1991-11-24,123.55,N
*/

Renaming column headers

Converting CSV column headers to JS property names

Of course people rarely use JavaScript property names for column headers. You would likely see "Date of birth" in CSV file header instead of "dob". The lil-csv is little but powerful. It can can handle that too.

That's how you can rename headers during CSV file parsing and CSV file generation.

You file:

First name,Last name,Date of birth,Price in dollars,Completed
John,Smith,1999-01-15,123.00,Y
Alice,Dwarf,1991-11-24,123.00,N

Renaming each column to a JS object property:

const people = parse(fileContents, {
  header: {
    "First name": "firstName",
    "Last name": "lastName",
    "Date of birth": {
      newName: "dob",
      parse: v => v ? new Date(v) : null,
    },
    "Price in dollars": {
      newName: "price",
      parse: v => isNaN(v) ? null : Number(v),
    },
    Completed: {
      newName: "completed",
      parse: v => String(v).toUpperCase() === "Y",
    },
  }
});

console.log(people);

/*
[
  {
    "firstName": "John",
    "lastName": "Smith",
    "dob": "1999-01-15T00:00:00.000Z",
    "price": 123.00,
    "completed": true
  },
  {
    "firstName": "Alice",
    "lastName": "Dwarf",
    "dob": "1991-11-24T00:00:00.000Z",
    "price": 123.00,
    "completed": false
  }
]
*/

Renaming JS properties to real world column headers

I hope this code is easy to read:

const string = generate(people, {
  header: {
    firstName: "First name",
    lastName: "Last name",
    dob: {
      newName: "Date of birth",
      stringify: v => v ? new Date(v).toISOString().substr(0, 10) : "",
    },
    price: {
      newName: "Price in dollars",
      stringify: v => isNaN(v) ? "" : Number(v).toFixed(2),
    },
    completed: {
      newName: "Completed",
      stringify: v => v ? "Y" : "N",
    },
  }
});

console.log(string);

/*
First name,Last name,Date of birth,Price in dollars,Completed
John,Smith,1999-01-15,123.00,Y
Alice,Dwarf,1991-11-24,123.00,N
*/

Deep objects support!

Here comes the true powers of lil-csv. You can parse CSV rows directly to deep objects like:

{
   order_id: 51234,
   recipient: {
     firstName: "John",
     lastName: "Smith",
     dob: "1999-01-15T00:00:00.000Z",
     address: {
       street: "123 John St, CARLTON",
       country: "AU",
     }
   }
}

Parsing CSV rows as JS deep objects

Let's parse this CSV to the above object:

ID,First name,Last name,Date of birth,Address,Country
51234,John,Smith,1999-01-15,"123 John St, CARLTON",AU

All you need is to rename headers with dot notation:

const orders = parse(fileContents, {
  header: {
    ID: {
      parse: Number,
      newName: "order_id",
    },
    "First name": "recipient.firstName",
    "Last name": "recipient.lastName",
    "Date of birth": {
      newName: "recipient.dob",
      parse: v => v ? new Date(v) : null,
    },
    Address: "recipient.address.street",
    Country: "recipient.address.country",
  }
});

console.log(orders);

It works similar when generating a CSV file from deep data:

const string = generate(orders, {
  header: {
    order_id: "ID",
    "recipient.firstName": "First name",
    "recipient.lastName": "Last name",
    "recipient.dob": {
      newName: "Date of birth",
      stringify: v => v ? new Date(v).toISOString().substr(0, 10) : "",
    },
    "recipient.address.street": "Address",
    "recipient.address.country": "Country",
  }
});

console.log(string);

/*
ID,First name,Last name,Date of birth,Address,Country
51234,John,Smith,1999-01-15,"123 John St, CARLTON",AU
*/

In the above code the "Date of birth" column gets:

  1. deeply set/accessed within an object.
  2. parsed/stringified in a custom way;
  3. renamed;

Afterword

You get all that power from 1 TCP packet, meaning less than 1460 bytes. Or even fewer bytes if you are using only one of the two functions (treeshaking is supported by lil-csv).

If you need additional features from lil-csv then feel free open an issue here: https://github.com/flash-oss/lil-csv/issues

17

Kategoriler:

Javascript

Tags:

JavascriptCsvNodeNpm