jsforce bulk query api 10k limit

Question

I am trying to understand how I can use jsforce & bulk query to export 50k records from salesforce. Only the first 10k is returned, I understand that this is due to a batch size limit of 10k however I don't understand how to create the next batch to get records 10001 through 20000 and so on.

Currently, I have the following, any help would be much appreciated.

conn.bulk.query('SELECT Id FROM Account')

    .on('record', function record(rec) {

      log.debug('dumpAllObject', 'rec', rec);

    })

    .on('error', function handle(err) { log.error('dumpAllObject', 'error', err); })

    .on('end', function resolve() {

          log.info('dumpAllObject', 'Completed');

    });

Answer 1

Go ahead and try using the authFetch and maxFetch options:

conn.query('SELECT Id FROM Account')

    .on('record', function record(rec) {

      log.debug('dumpAllObject', 'rec', rec);

    })

    .on('error', function handle(err) { log.error('dumpAllObject', 'error', err); })

    .on('end', function resolve() {

          log.info('dumpAllObject', 'Completed');

    })

    .run({

        authFetch: true,

        maxFetch: 20000

    });

This should work. I'm not sure what the difference between conn.query and conn.bulk.query is.

Look at the Salesforce Course provided by Intellipaat!