API

Modules

CozyBrowser: Get a javascript simulation of a real browser (jsdom)
addData: Saves the data into the cozy blindly without check.
cozyClient: cozy-client-js instance already initialized and ready to use.
hydrateAndFilter: Filters the passed array from data already present in the cozy so that there is not duplicated data in the Cozy.
linkBankOperations: Finds links between bills and bank operations.
mkdirp
normalizeFilename: Returns the given name, replacing characters that could be an issue when used in a filename with spaces.
saveBills: Encapsulates the saving of Bills : saves the files, saves the new data, and associate the files to an existing bank operation
saveFiles: Saves the given files in the given folder via the Cozy API.
saveIdentity: Helper to set or merge io.cozy.identities See https://github.com/cozy/cozy-doctypes/blob/master/docs/io.cozy.identities.md
signin: Provides an handy method to log the user in, on HTML form pages. On success, it resolves to a promise with a parsed body.
solveCaptcha: Use every possible means to solve a captcha. At the moment, Anticaptcha web service is used if any related secret key is found in COZY_PARAMETERS environment variable.
updateOrCreate: Creates or updates the given entries according to if they already exist in the cozy or not
utils: Small utilities helping to develop konnectors.
categorization: Bank transactions categorization

Classes

The class from which all the connectors must inherit. It takes a fetch function in parameter that must return a Promise. You need at least the GET permission on io.cozy.accounts in your manifest to allow it to fetch account information for your connector.

Its role is twofold :

Make the link between account data and konnector
Handle errors

⚠️ A promise should be returned from the fetch function otherwise the konnector cannot know that asynchronous code has been called.

CookieKonnector

Connector base class extending BaseKonnector which handles cookie session in a central way It also handles saving cookie session in the account and automatically restore it for the next connector run. All cozy-konnector-libs tools using request are proposed as methods of this class to force them to use the central cookie which can be saved/restored. You need at least the GET and PUT permissions on io.cozy.accounts in your manifest to allow it to save/restore cookies

Document

Simple Model for Documents. Allows to specify shouldSave, shouldUpdate as methods.

Has useful isEqual method

Constants

fs

Manifest is provided differently in developement that in production.

In production, the manifest has been "merged" via Webpack via the DefinePlugin
In development/test, we simply read the manifest from the fs

LOGIN_FAILED : string

The konnector could not login

NOT_EXISTING_DIRECTORY : string

The folder specified as folder_to_save does not exist (checked by BaseKonnector)

VENDOR_DOWN : string

The vendor's website is down

USER_ACTION_NEEDED : string

There was an unexpected error, please take a look at the logs to know what happened

FILE_DOWNLOAD_FAILED : string

There was a problem while downloading a file

SAVE_FILE_FAILED : string

There was a problem while saving a file

DISK_QUOTA_EXCEEDED : string

Could not save a file to the cozy because of disk quota exceeded

CHALLENGE_ASKED : string

It seems that the website requires a second authentification factor that we don’t support yet.

LOGIN_FAILED_TOO_MANY_ATTEMPTS : string

Temporarily blocked

USER_ACTION_NEEDED_OAUTH_OUTDATED : string

Access refresh required

USER_ACTION_NEEDED_ACCOUNT_REMOVED : string

Unavailable account

USER_ACTION_NEEDED_CHANGE_PASSWORD : string

Unavailable account

USER_ACTION_NEEDED_PERMISSIONS_CHANGED : string

Password update required

USER_ACTION_NEEDED_CGU_FORM : string

The user needs to accept a CGU form before accessing the rest of the website

CAPTCHA_RESOLUTION_FAILED : string

solveCaptcha failed to solve the captcha

LOGIN_FAILED_NEEDS_SECRET : string

Additionnal information is needed to check your login details

MAINTENANCE : string

remote website seems to be unavailable

TERMS_VERSION_MISMATCH : string

User needs to accept new terms

UNKNOWN_ERROR : string

unkown error

UNKNOWN_ERROR_PARTIAL_SYNC : string

The synchronization is complete but some elements may be missing

USER_ACTION_NEEDED_SCA_REQUIRED : string

Renewal of authentication required

USER_ACTION_NEEDED_TWOFA_EXPIRED : string

Authentication renewal required

USER_ACTION_NEEDED_WEBAUTH_REQUIRED : string

Authentication on vendor website required

USER_ACTION_NEEDED_WRONG_TWOFA_CODE : string

Incorrect strong authentication code

VENDOR_DOWN_BANK_DOWN : string

Unavailable bank website

VENDOR_DOWN_LINXO_DOWN : string

Unavailable bank website

Functions

attachProcessEventHandlers(prcs) ⇒ function

Attach event handlers to catch uncaught exceptions/rejections and signals. Log them as critical and exit the process accordingly. If the cleanup function has not been called, calling again the function is a no-op.

mkSpec()

Declarative scraping.

Describe your items attributes and where to find/parse them instead of imperatively building them.

Heavily inspired by artoo scraping method.

scrape($, specs, [childSelector]) ⇒ object | Array

Scrape a cheerio object for properties

CozyBrowser

Get a javascript simulation of a real browser (jsdom)

CozyBrowser
- defaultOptions ⇒ object ⏏
  - ~addListeners()

defaultOptions ⇒ `object` ⏏

Get a preconfigured jsdom browser simulator using the zombie npm package See http://zombie.js.org/ for complete documentation The connector has to import the zombie npm package itself.

Kind: Exported constant
Returns: object - Zombie browser extended class

Param	Type	Description
options.userAgent	`string`	The user agent string used by the browser

Example

const Browser = require('cozy-konnector-libs/libs/CozyBrowser')
const browser = new Browser()
await browser.visit('http://quotes.toscrape.com/')

defaultOptions~addListeners()

Add cozy-konnector-libs specific logs to browser events

Kind: inner method of defaultOptions

addData

Saves the data into the cozy blindly without check.

addData() ⏏

Saves the data into the cozy blindly without check.

You need at least the POST permission for the given doctype in your manifest, to be able to use this function.

Parameters:

documents: an array of objects corresponding to the data you want to save in the cozy doctype (string): the doctype where you want to save data (ex: 'io.cozy.bills') options (object): option object

sourceAccount (String): id of the source account
sourceAccountIdentifier (String): identifier unique to the account targetted by the connector. It is the login most of the time

const documents = [
  {
    name: 'toto',
    height: 1.8
  },
  {
    name: 'titi',
    height: 1.7
  }
]

return addData(documents, 'io.cozy.height')

Kind: Exported function

cozyClient

cozy-client-js instance already initialized and ready to use.

cozyClient ⏏

cozy-client-js instance already initialized and ready to use.

If you want to access cozy-client-js directly, this method gives you directly an instance of it, initialized according to COZY_URL and COZY_CREDENTIALS environment variable given by cozy-stack You can refer to the cozy-client-js documentation for more information.

Example :

const {cozyClient} = require('cozy-konnector-libs')

cozyClient.data.defineIndex('my.doctype', ['_id'])

Kind: Exported constant

hydrateAndFilter

Filters the passed array from data already present in the cozy so that there is not duplicated data in the Cozy.

hydrateAndFilter
- hydrateAndFilter() ⏏
  - ~suitableCall()

hydrateAndFilter() ⏏

Filters the passed array from data already present in the cozy so that there is not duplicated data in the Cozy.

You need at least the GET permission for the given doctype in your manifest, to be able to use this function.

Parameters:

documents: an array of objects corresponding to the data you want to save in the cozy doctype (string): the doctype where you want to save data (ex: 'io.cozy.bills') options :

keys (array) : List of keys used to check that two items are the same. By default it is set to `['id']'.
index (optionnal) : Return value returned by cozy.data.defineIndex, the default will correspond to all documents of the selected doctype.
selector (optionnal object) : Mango request to get records. Default is built from the keys {selector: {_id: {"$gt": null}}} to get all the records.

const documents = [
  {
    name: 'toto',
    height: 1.8
  },
  {
    name: 'titi',
    height: 1.7
  }
]

return hydrateAndFilter(documents, 'io.cozy.height', {
  keys: ['name']
}).then(filteredDocuments => addData(filteredDocuments, 'io.cozy.height'))

Kind: Exported function

hydrateAndFilter~suitableCall()

Since we can use methods or basic functions for shouldSave and shouldUpdate we pass the appropriate this and arguments.

If funcOrMethod is a method, it will be called with args[0] as this and the rest as arguments Otherwise, this will be null and args will be passed as arguments.

Kind: inner method of hydrateAndFilter

linkBankOperations

Finds links between bills and bank operations.

linkBankOperations() ⏏

Will soon move to a dedicated service. You should not use it.

Finds links between bills and bank operations.

Kind: Exported function

mkdirp

mkdirp ⏏

Creates a directory and its missing ancestors as needed.

Options :

...pathComponents: one or many path components to be joined

await mkdirp('/foo') // Creates /foo
await mkdirp('/foo') // Does nothing as /foo already exists
await mkdirp('/bar/baz') // Creates /bar, then /bar/baz
await mkdirp('/foo/bar/baz') // Creates /foo/bar, then /foo/bar/baz, not /foo
await mkdirp('/') // Does nothing
await mkdirp('/qux', 'qux2/qux3', 'qux4') // Creates /qux, then /qux/qux2,
                                          // then /qux/qux2/qux3 and
                                          // finally /qux/qux2/qux3/qux4

The function will automatically add a leading slash when missing:

await mkdirp('foo', 'bar') // Creates /foo, then /foo/bar

Kind: Exported constant

normalizeFilename

Returns the given name, replacing characters that could be an issue when used in a filename with spaces.

normalizeFilename() ⏏

Returns the given name, replacing characters that could be an issue when used in a filename with spaces.

Replaced characters include:

Those forbidden on one or many popular OS or filesystem: <>:"/\|?*
Those forbidden by the cozy-stack \0, \r and \n
Multiple spaces and/or tabs are replaced with a single space
Leading & trailing spaces and/or tabs are removed

An exception will be thrown in case there is not any filename-compatible character in the given name.

Parameters:

basename is whatever string you want to generate the filename from
ext is an optional file extension, with or without leading dot

const { normalizeFilename } = require('cozy-konnector-libs')

const filename = normalizeFilename('*foo/bar: <baz> \\"qux"\t???', '.txt')
// `filename` === `foo bar baz qux.txt`

Kind: Exported function

saveBills

Encapsulates the saving of Bills : saves the files, saves the new data, and associate the files to an existing bank operation

saveBills() ⏏

Combines the features of saveFiles, hydrateAndFilter, addData and linkBankOperations for a common case: bills. Will create io.cozy.bills objects. The default deduplication keys are ['date', 'amount', 'vendor']. You need the full permission on io.cozy.bills, full permission on io.cozy.files and also full permission on io.cozy.bank.operations in your manifest, to be able to use this function.

Parameters:

documents is an array of objects with any attributes with some mandatory attributes :
- amount (Number): the amount of the bill used to match bank operations
- date (Date): the date of the bill also used to match bank operations
- vendor (String): the name of the vendor associated to the bill. Ex: 'trainline'
- currency (String) default: EUR: The ISO currency value (not mandatory since there is a default value.
- contractId (String): Contract unique identicator used to deduplicate bills
- contractLabel: (String) User label if define, must be used with contractId
- matchingCriterias (Object): criterias that can be used by an external service to match bills with bank operations. If not specified but the 'banksTransactionRegExp' attribute is specified in the manifest of the connector, this value is automatically added to the bill
You can also pass attributes expected by saveFiles : fileurl, filename, requestOptions and more

Please take a look at io.cozy.bills doctype documentation
fields (object) this is the first parameter given to BaseKonnector's constructor
options is passed directly to saveFiles, hydrateAndFilter, addData and linkBankOperations.

Kind: Exported function
Example

const { BaseKonnector, saveBills } = require('cozy-konnector-libs')

module.exports = new BaseKonnector(function fetch (fields) {
  const documents = []
  // some code which fills documents
  return saveBills(documents, fields, {
    identifiers: ['vendor']
  })
})

saveFiles

Saves the given files in the given folder via the Cozy API.

saveFiles(entries, fields, options, [contract]) ⏏

Saves the files given in the fileurl attribute of each entries

You need the full permission on io.cozy.files in your manifest to use this function.

Kind: Exported function

Param	Type	Description
entries	`Array`	list of object describing files to save
entries.fileurl	`string`	The url of the file (can be a function returning the value). Ignored if `filestream` is given
entries.fetchFile	`function`	the connector can give it's own function to fetch the file from the website, which will be run only when necessary (if the corresponding file is missing on the cozy) function returning the stream). This function must return a promise resolved as a stream
entries.filestream	`object` \| `string`	the stream which will be directly passed to cozyClient.files.create (can also be function returning the stream)
entries.requestOptions	`object`	The options passed to request to fetch fileurl (can be a function returning the value)
entries.filename	`string`	The file name of the item written on disk. This attribute is optional and as default value, the file name will be "smartly" guessed by the function. Use this attribute if the guess is not smart enough for you, or if you use `filestream` (can be a function returning the value).
entries.shouldReplaceName	`string`	used to migrate filename. If saveFiles finds a file linked to this entry and this file name matches `shouldReplaceName`, the file is renamed to `filename` (can be a function returning the value)
entries.shouldReplaceFile	`function`	use this function to state if the current entry should be forced to be redownloaded and replaced. Usefull if we know the file content can change and we always want the last version.
entries.fileAttributes	`object`	ex: `{created_at: new Date()}` sets some additionnal file attributes passed to cozyClient.file.create
entries.subPath	`string`	A subpath to save all files, will be created if needed.
entries.contract	`object`	contract object associated to the files
entries.contract.id	`string`	id of the contract
entries.contract.name	`string`	name of the contract
fields	`object`	is the argument given to the main function of your connector by the BaseKonnector. It especially contains a `folderPath` which is the string path configured by the user in collect/home
options	`object`	global options
options.timeout	`number`	timestamp which can be used if your connector needs to fetch a lot of files and if the stack does not give enough time to your connector to fetch it all. It could happen that the connector is stopped right in the middle of the download of the file and the file will be broken. With the `timeout` option, the `saveFiles` function will check if the timeout has passed right after downloading each file and then will be sure to be stopped cleanly if the timeout is not too long. And since it is really fast to check that a file has already been downloaded, on the next run of the connector, it will be able to download some more files, and so on. If you want the timeout to be in 10s, do `Date.now() + 10*1000`. You can try it in the previous code.
options.contentType	`number` \| `boolean`	ex: 'application/pdf' used to force the contentType of documents when they are badly recognized by cozy. If "true" the content type will be recognized from the file name and forced the same way.
options.concurrency	`number`	default: `1` sets the maximum number of concurrent downloads
options.validateFile	`function`	default: do not validate if file is empty or has bad mime type
options.validateFileContent	`boolean` \| `function`	default false. Also check the content of the file to recognize the mime type
options.fileIdAttributes	`Array`	array of strings : Describes which attributes of files will be taken as primary key for files to check if they already exist, even if they are moved. If not given, the file path will used for deduplication as before.
options.subPath	`string`	A subpath to save this file, will be created if needed.
[contract]	`object`	contract object associated to the file
[contract.id]	`string`	id of the contract
[contract.name]	`string`	name of the contract
options.fetchFile	`function`	the connector can give it's own function to fetch the file from the website, which will be run only when necessary (if the corresponding file is missing on the cozy) function returning the stream). This function must return a promise resolved as a stream
options.verboseFilesLog	`boolean`	the connector will send saveFiles result as a warning

Example

await saveFiles([{fileurl: 'https://...', filename: 'bill1.pdf'}], fields, {
   fileIdAttributes: ['fileurl']
})

saveIdentity

Helper to set or merge io.cozy.identities See https://github.com/cozy/cozy-doctypes/blob/master/docs/io.cozy.identities.md

saveIdentity() ⏏

Set or merge a io.cozy.identities

You need full permission for the doctype io.cozy.identities in your manifest, to be able to use this function. Parameters:

identity (object): the identity to create/update as an io.cozy.identities object accountIdentifier (string): a string that represent the account use, if available fields.login options (object): options which will be given to updateOrCreate directly :

sourceAccount (String): id of the source account
sourceAccountIdentifier (String): identifier unique to the account targetted by the connector. It is the login most of the time

const { saveIdentity } = require('cozy-konnector-libs')
const identity =
  {
    contact: {
      name: 'toto',
      email: { 'address': 'toto@example.com' }
    }
  }

return saveIdentity(identity, fields.login)

Kind: Exported function

signin

Provides an handy method to log the user in, on HTML form pages. On success, it resolves to a promise with a parsed body.

signin() ⏏

Provides an handy method to log the user in, on HTML form pages. On success, it resolves to a promise with a parsed body.

Errors:

LOGIN_FAILED if the validate predicate is false
INVALID_FORM if the element matched by formSelector is not a form or has no action attribute
UNKNOWN_PARSING_STRATEGY if parse is not one of the accepted values: raw, cheerio, json.
VENDOR_DOWN if a request throws a RequestError, or StatusCodeError

It does not submit values provided through select tags, except if populated by user with formData.

url is the url to access the html form
formSelector is used by cheerio to uniquely identify the form in which to log in
formData is an object { name: value, … }. It is used to populate the form, in the proper inputs with the same name as the properties of this object, before submitting it. It can also be a function that returns this object. The page at url would be given as argument, right after having been parsed through cheerio.
parse allow the user to resolve signin with a preparsed body. The choice of the strategy for the parsing is one of : raw, json or cheerio. cheerio being the default.
validate is a predicate taking three arguments statusCode, parsedBody and fullResponse. If it is false, LOGIN_FAILED is thrown, otherwise the signin resolves with parsedBody value.
requestOpts allows to pass eventual options to the signin's requestFactory. It could be useful for pages using latin1 encoding for instance.

Kind: Exported function
Example
== basic example : ==

const $ = signin({
  url: `http://quotes.toscrape.com/login`,
  formSelector: 'form',
  formData: { username, password }
})

If the behavior of the targeted website is not standard. You can pass a validate function which will allow you to:

detect if the credentials work or not -> LOGIN_FAILED
detect if actions from the user are needed -> USER_ACTION_NEEDED
detect if the targeted website is out -> VENDOR_DOWN Example

const $ = signin({
  url: `http://quotes.toscrape.com/login`,
  formSelector: 'form',
  formData: { username, password },
  validate: (statusCode, $, fullResponse) {
   if (statusCode !== 200) return false // LOGIN_FAILED
   if ($('.cgu').length) throw new Error('USER_ACTION_NEEDED')
   if (fullResponse.request.uri.href.includes('error')) throw new Error('VENDOR_DOWN')
  }
})

Do not forget that the use of the signin function is not mandatory in a connector and won't work if the signin page does not use html forms. Here, a simple POST request may be a lot more simple.

solveCaptcha

Use every possible means to solve a captcha. At the moment, Anticaptcha web service is used if any related secret key is found in COZY_PARAMETERS environment variable.

solveCaptcha() ⏏

Use every possible means to solve a captcha. At the moment, Anticaptcha web service is used if any related secret key is found in COZY_PARAMETERS environment variable. If you do not want to solve the captcha each time the connector is run, please also use CookieKonnector which will help you save the session.

Parameters:

params is an array of objects with any attributes with some mandatory attributes :
- type (String): (default recaptcha) type of captcha to solve. can be "recaptcha" or "image" at the moment
- timeout (Number): (default 3 minutes after now) time when the solver should stop trying to solve the captcha
- withFullSolution(Boolean): (default false) Change the return to an object containing full solution
- websiteKey (String): the key you can find on the targeted website (for recaptcha)
- websiteURL (String): The URL of the page showing the captcha (for recaptcha)
- body (String): The base64 encoded image (for image captcha) Returns: Promise with the solved captcha response as a string

Kind: Exported function
Example

const { solveCaptcha } = require('cozy-konnector-libs')

const solvedKey = await solveCaptcha({
  websiteKey: 'the key in the webpage',
  websiteURL: 'http://quotes.toscrape.com/login',
})
// now use the solveKey to submit your form

updateOrCreate

Creates or updates the given entries according to if they already exist in the cozy or not

updateOrCreate() ⏏

Creates or updates the given entries according to if they already exist in the cozy or not

You need the full permission for the given doctype in your manifest, to be able to use this function.

entries (object array): Documents to save doctype (string): Doctype of the documents matchingAttributes (string array): attributes in each entry used to check if an entry already exists in the Cozy options (object): general option affecting metadata :

sourceAccount (String): id of the source account
sourceAccountIdentifier (String): identifier unique to the account targetted by the connector. It is the login most of the time

Kind: Exported function

utils

Small utilities helping to develop konnectors.

utils

utils~fetchAll()

This function allows to fetch all documents for a given doctype. It is the fastest to get all documents but without filtering possibilities deprecated by the findAll method from cozyClient

Parameters:

doctype (string): the doctype from which you want to fetch the data

Kind: inner method of utils

utils~queryAll()

This function allows to fetch all documents for a given doctype exceeding the 100 limit. It is slower that fetchAll because it fetches the data 100 by 100 but allows to filter the data with a selector and an index

Parameters:

doctype (string): the doctype from which you want to fetch the data selector (object): the mango query selector index (object): (optional) the query selector index. If not defined, the function will create it's own index with the keys specified in the selector

const documents = await queryAll('io.cozy.bills', {vendor: 'Direct Energie'})

Kind: inner method of utils

utils~findDuplicates()

This function find duplicates in a given doctype, filtered by an optional mango selector

Parameters:

doctype (string): the doctype from which you want to fetch the data selector (object): (optional) the mango query selector options :

keys (array) : List of keys used to check that two items are the same.
index (optionnal) : Return value returned by cozy.data.defineIndex, the default will correspond to all documents of the selected doctype.
selector (optionnal object) : Mango request to get records. Gets all the records by default

Returns an object with the following keys: toKeep: this is the list of unique documents that you should keep in db toRemove: this is the list of documents that can remove from db. If this is io.cozy.bills documents, do not forget to clean linked bank operations

const {toKeep, toRemove} = await findDuplicates('io.cozy.bills', {selector: {vendor: 'Direct Energie'}})

Kind: inner method of utils

utils~batchUpdateAttributes()

This is a shortcut to update multiple documents in one call

Parameters:

doctype (string): the doctype from which you want to fetch the data ids (array): array of ids of documents to update transformation (object): attributes to change with their values options :

keys (array) : List of keys used to check that two items are the same.
index (optionnal) : Return value returned by cozy.data.defineIndex, the default will correspond to all documents of the selected doctype.
selector (optionnal object) : Mango request to get records. Gets all the records by default

Returns a promise which resolves with all the return values of updateAttributes

await batchUpdateAttributes('io.cozy.bills', [1, 2, 3], {vendor: 'Direct Energie'})

Kind: inner method of utils

utils~batchDelete()

This is a shortcut to delete multiple documents in one call

Parameters:

doctype (string): the doctype from which you want to fetch the data documents (array): documents to delete with their ids transformation (object): attributes to change with their values options :

keys (array) : List of keys used to check that two items are the same.
index (optionnal) : Return value returned by cozy.data.defineIndex, the default will correspond to all documents of the selected doctype.
selector (optionnal object) : Mango request to get records. Gets all the records by default

Returns a promise which resolves with all the return values of updateAttributes

Example to remove all the documents for a given doctype

const documents = await fetchAll('io.cozy.marvel')
await batchDelete('io.cozy.marvel', documents)

Kind: inner method of utils

utils~getPdfText()

This function can read the content of a cozy pdf file and output its text

Parameters:

fileId (string): the id of the file in the cozy options :

pages (array or number) : The list of page you want to interpret

You need to add pdfjs-dist package as a dependency to your connector to allow this to work

Returns a promise which resolves with an object with the following attributes:

text (string) : The full text of the pdf
1 : The full pdfjs data for page 1
n : The full pdfjs data for page n

Example:

const pdfText = (await getPdfText('887ABCFE87687')).text

Kind: inner method of utils

utils~formatDate()

This function convert a Date Object to a ISO date string (2018-07-31)

Parameters:

date (Date): the id of the file in the cozy

Returns a string

Example:

const date = formatFrenchDate(New Date.now())

Kind: inner method of utils

categorization

Bank transactions categorization

categorization
- ~createCategorizer(options) ⇒ Object
- ~categorize(transactions, options) ⇒ Array.<object>
- ~CreateCategorizerOptions

categorization~createCategorizer(options) ⇒ `Object`

Initialize global and local models and return an object exposing a categorize function that applies both models on an array of transactions

The global model is a model specific to hosted Cozy instances. It is not available for self-hosted instances. It will just do nothing in that case.

The local model is based on the user manual categorizations.

Each model adds two properties to the transactions: The global model adds cozyCategoryId and cozyCategoryProba The local model adds localCategoryId and localCategoryProba

In the end, each transaction can have up to four different categories. An application can use these categories to show the most significant for the user. See https://github.com/cozy/cozy-doctypes/blob/master/docs/io.cozy.bank.md#categories for more informations.

Kind: inner method of categorization
Returns: Object - A method to categorize transactions and the classifiers it uses.

Param	Type	Description
options	`CreateCategorizerOptions`	Options used to build the categorizer

Example

const { BaseKonnector, createCategorizer } = require('cozy-konnector-libs')

class BankingKonnector extends BaseKonnector {
  async saveTransactions() {
    const transactions = await this.fetchTransactions()
    const categorizer = await createCategorizer
    const categorizedTransactions = await categorizer.categorize(transactions)

    // Save categorizedTransactions
  }
}

categorization~categorize(transactions, options) ⇒ `Array.<object>`

Initialize global and local models and categorize the given array of transactions

Kind: inner method of categorization
Returns: Array.<object> - the categorized transactions
See: createCategorizer for more informations about models initialization

Param	Type	Description
transactions	`Array.<object>`	The transactions to categorize
options	`CreateCategorizerOptions`	Options passed to create the categorizer

Example

const { BaseKonnector, categorize } = require('cozy-konnector-libs')

class BankingKonnector extends BaseKonnector {
  async saveTransactions() {
    const transactions = await this.fetchTransactions()
    const categorizedTransactions = await categorize(transactions)

    // Save categorizedTransactions
  }
}

categorization~CreateCategorizerOptions

Kind: inner typedef of categorization
Properties

Name	Type	Description
useGlobalModel	`boolean`	Whether to use the globally trained model
customTransactionFetcher	`function`	A custom training transaction fetcher
pretrainedClassifier	`object`	A pretrained instance of a bayes classifier

BaseKonnector

The class from which all the connectors must inherit. It takes a fetch function in parameter that must return a Promise. You need at least the GET permission on io.cozy.accounts in your manifest to allow it to fetch account information for your connector.

Its role is twofold :

Make the link between account data and konnector
Handle errors

⚠️ A promise should be returned from the fetch function otherwise the konnector cannot know that asynchronous code has been called.

Kind: global class

BaseKonnector
- new BaseKonnector(fetch)
- .run()
- .main() ⇒ Promise
- .end()
- .fail()
- .readPayload() ⇒ Object | null
- .initAttributes()
- .saveAccountData(data, options) ⇒ Promise
- .getAccountData() ⇒ object
- .updateAccountAttributes()
- .setTwoFAState(options)
- .resetTwoFAState()
- .waitForTwoFaCode(options) ⇒ Promise
- .notifySuccessfulLogin()
- .deactivateAutoSuccessfulLogin()
- .saveBills() ⇒ Promise
- .saveFiles() ⇒ Promise
- .updateOrCreate() ⇒ Promise
- .saveIdentity() ⇒ Promise
- .signin() ⇒ Promise
- .terminate(err)
- .getCozyMetadata(data)

new BaseKonnector(fetch)

Constructor

Param	Type	Description
fetch	`function`	Function to be run automatically after account data is fetched. This function will be binded to the current connector. If not fetch function is given. The connector will have to handle itself it's own exection and error handling

Example

const { BaseKonnector } = require('cozy-konnector-libs')

module.exports = new BaseKonnector(function fetch () {
 // use this to access the instance of the konnector to
 // store any information that needs to be passed to
 // different stages of the konnector
 return request('http://ameli.fr')
   .then(computeReimbursements)
   .then(saveBills)
})

baseKonnector.run()

Entrypoint of the konnector

Initializes connector attributes
Awaits this.main
Ensures errors are handled via this.fail
Calls this.end when the main function succeeded

Kind: instance method of BaseKonnector

baseKonnector.main() ⇒ `Promise`

Main runs after konnector has been initialized. Errors thrown will be automatically handled.

Kind: instance method of BaseKonnector
Returns: Promise - - The konnector is considered successful when it resolves

baseKonnector.end()

Hook called when the connector has ended successfully

Kind: instance method of BaseKonnector

baseKonnector.fail()

Hook called when the connector fails

Kind: instance method of BaseKonnector

baseKonnector.readPayload() ⇒ `Object` | `null`

Read an eventual payload from COZY_PAYLOAD env var, wether it is a JSON string or a reference to a file containing a JSON string

Kind: instance method of BaseKonnector
Returns: Object | null - Promise<> result of JSON.parse from the JSON string or null if no payload

baseKonnector.initAttributes()

Initializes konnector attributes that will be used during its lifetime

this._account
this.fields

Kind: instance method of BaseKonnector

baseKonnector.saveAccountData(data, options) ⇒ `Promise`

Saves data to the account that is passed to the konnector. Use it to persist data that needs to be passed to each konnector run.

By default, the data is merged to the remote data, use options.merge = false to overwrite the data.

The data is saved under the .data attribute of the cozy account.

Don't forget to modify the manifest.konnector file to give the right to write on the io.cozy.accounts doctype. The syntax can be : "permissions": {"accounts": {"type": "io.cozy.accounts"}} (here we juste removed the verb GET)

Kind: instance method of BaseKonnector
Returns: Promise - : resolved with the modified account

Param	Type	Description
data	`object`	Attributes to be merged
options	`object`	{ merge: true

baseKonnector.getAccountData() ⇒ `object`

Get the data saved by saveAccountData

Kind: instance method of BaseKonnector
Returns: object - the account data

baseKonnector.updateAccountAttributes()

Update account attributes and cache the account

Kind: instance method of BaseKonnector

baseKonnector.setTwoFAState(options)

Sets the 2FA state, according to the type passed. Doing so resets the twoFACode field

Typically you should not use that directly, prefer to use waitForTwoFaCode since the wait for user input will be handled for you. It is useful though for the "app" type where no user input (inside Cozy) is needed.

Kind: instance method of BaseKonnector

Param	Type	Description
options	`object`	The list of options
options.type	`string`	Used by the front to show the right message (email/sms/app/app_code)
options.retry	`boolean`	Is this function call a retry ? This changes the resulting message to the user

baseKonnector.resetTwoFAState()

Resets 2FA state when not needed anymore

Kind: instance method of BaseKonnector

baseKonnector.waitForTwoFaCode(options) ⇒ `Promise`

Notices that 2FA code is needed and wait for the user to submit it. It uses the account to do the communication with the user.

Kind: instance method of BaseKonnector
Returns: Promise - Contains twoFa code entered by user
Throws:

Will throw USER_ACTION_NEEDED.TWOFA_EXPIRED if the konnector job is not run manually (we assume that not run manually means that we do not have a graphic interface to fill the required information)
Will throw USER_ACTION_NEEDED.TWOFA_EXPIRED if 2FA is not filled by the user soon enough

Param	Type	Description
options	`object`	The list of options
options.type	`string`	(default: "email") - Type of the expected 2FA code. The message displayed to the user will depend on it. Possible values: email, sms
options.timeout	`number`	(default 3 minutes after now) - After this date, the stop will stop waiting and and an error will be shown to the user (deprecated and alias of endTime)
options.endTime	`number`	(default 3 minutes after now) - After this timestamp, the home will stop waiting and and an error will be shown to the user
options.heartBeat	`number`	(default: 5000) - How many milliseconds between each code check
options.retry	`boolean`	(default: false) - Is it a retry. If true, an error message will be displayed to the user

Example

const { BaseKonnector } = require('cozy-konnector-libs')

module.exports = new BaseKonnector(start)
async function start() {
   // we detect the need of a 2FA code
   const code = this.waitForTwoFaCode({
     type: 'email'
   })
   // send the code to the targeted site
}

baseKonnector.notifySuccessfulLogin()

Tells Cozy-Home that we have successfully logged in. Useful when auto-success has been deactivated. See deactivateAutoSuccess

Kind: instance method of BaseKonnector

baseKonnector.deactivateAutoSuccessfulLogin()

By default, cozy-home considers that the konnector has successfully logged in when the konnector has run for more than 8s. This is problematic for 2FA since the konnector can sit idle, just waiting for the 2FA to come back.

When this method is called, cozy-home is notified and will not consider the absence of error after 8s to be a success. Afterwards, to notify cozy-home when the user has logged in successfully, for example, after the user has entered 2FA codes, it is necessary to call notifySuccessfulLogin.

Does nothing if called more than once.

Kind: instance method of BaseKonnector

baseKonnector.saveBills() ⇒ `Promise`

This is saveBills function from cozy-konnector-libs which automatically adds sourceAccount in metadata of each entry

Kind: instance method of BaseKonnector
Returns: Promise - resolves with entries hydrated with db data

baseKonnector.saveFiles() ⇒ `Promise`

This is saveFiles function from cozy-konnector-libs which automatically adds sourceAccount and sourceAccountIdentifier cozyMetadatas to files

Kind: instance method of BaseKonnector
Returns: Promise - resolves with the list of entries with file objects

baseKonnector.updateOrCreate() ⇒ `Promise`

This is updateOrCreate function from cozy-konnector-libs which automatically adds sourceAccount in metadata of each entry

Kind: instance method of BaseKonnector
Returns: Promise - resolves to an array of db objects

baseKonnector.saveIdentity() ⇒ `Promise`

This is saveIdentity function from cozy-konnector-libs which automatically adds sourceAccount in metadata of each entry

Kind: instance method of BaseKonnector
Returns: Promise - empty promise

baseKonnector.signin() ⇒ `Promise`

This is signin function from cozy-konnector-libs which automatically adds deactivateAutoSuccessfulLogin and notifySuccessfulLogin calls

Kind: instance method of BaseKonnector
Returns: Promise - resolve with an object containing form data

baseKonnector.terminate(err)

Send a special error code which is interpreted by the cozy stack to terminate the execution of the connector now

Kind: instance method of BaseKonnector

Param	Type	Description
err	`string`	The error code to be saved as connector result see [docs/ERROR_CODES.md]

Example

this.terminate('LOGIN_FAILED')

baseKonnector.getCozyMetadata(data)

Get cozyMetaData from the context of the connector

Kind: instance method of BaseKonnector

Param	Type	Description
data	`object`	this data will be merged with cozyMetaData

CookieKonnector

Connector base class extending BaseKonnector which handles cookie session in a central way It also handles saving cookie session in the account and automatically restore it for the next connector run. All cozy-konnector-libs tools using request are proposed as methods of this class to force them to use the central cookie which can be saved/restored. You need at least the GET and PUT permissions on io.cozy.accounts in your manifest to allow it to save/restore cookies

Kind: global class

CookieKonnector
- new CookieKonnector(requestFactoryOptions)
- .initAttributes() ⇒ Promise
- .end()
- .requestFactory(options) ⇒ object
- .resetSession() ⇒ Promise
- .initSession() ⇒ Promise
- .saveSession() ⇒ Promise
- .signin() ⇒ Promise
- .saveFiles() ⇒ Promise
- .saveBills() ⇒ Promise

new CookieKonnector(requestFactoryOptions)

Constructor

Param	Type	Description
requestFactoryOptions	`function`	Option object passed to requestFactory to initialize this.request. It is still possible to change this.request doing : `javascript this.request = this.requestFactory(...)` Please not you have to run the connector yourself doing : `javascript connector.run()`

Example

const { CookieKonnector } = require('cozy-konnector-libs')
class MyConnector extends CookieKonnector {
  async fetch(fields) {
     // the code of your connector
     await this.request('https://...')
  }
  async testSession() {
     const $ = await this.request('https://...')
     return $('')
  }
}
const connector = new MyKonnector({
  cheerio: true,
  json: false
})
connector.run()

cookieKonnector.initAttributes() ⇒ `Promise`

Initializes the current connector with data coming from the associated account and also the session

Kind: instance method of CookieKonnector
Returns: Promise - with the fields as an object

cookieKonnector.end()

Hook called when the connector is ended

Kind: instance method of CookieKonnector

cookieKonnector.requestFactory(options) ⇒ `object`

Calls cozy-konnector-libs requestFactory forcing this._jar as the cookie

Kind: instance method of CookieKonnector
Returns: object - - The resulting request object

Param	Type	Description
options	`object`	requestFactory option

cookieKonnector.resetSession() ⇒ `Promise`

Reset cookie session with a new empty session and save it to the associated account

Kind: instance method of CookieKonnector
Returns: Promise - empty promise

cookieKonnector.initSession() ⇒ `Promise`

Get the cookie session from the account if any

Kind: instance method of CookieKonnector
Returns: Promise - true or false if the session in the account exists or not

cookieKonnector.saveSession() ⇒ `Promise`

Saves the current cookie session to the account

Kind: instance method of CookieKonnector
Returns: Promise - empty promise

cookieKonnector.signin() ⇒ `Promise`

This is signin function from cozy-konnector-libs which is forced to use the current cookies and current request from CookieKonnector. It also automatically saves the session after signin if it is a success.

Kind: instance method of CookieKonnector
Returns: Promise - resolve with an object containing form data

cookieKonnector.saveFiles() ⇒ `Promise`

This is saveFiles function from cozy-konnector-libs which is forced to use the current cookies and current request from CookieKonnector.

Kind: instance method of CookieKonnector
Returns: Promise - resolves with the list of entries with file objects

cookieKonnector.saveBills() ⇒ `Promise`

This is saveBills function from cozy-konnector-libs which is forced to use the current cookies and current request from CookieKonnector.

Kind: instance method of CookieKonnector
Returns: Promise - resolves with entries hydrated with db data

Document

Simple Model for Documents. Allows to specify shouldSave, shouldUpdate as methods.

Has useful isEqual method

Kind: global class

document.isEqual()

Compares to another document deeply.

_id and _rev are by default ignored in the comparison.

By default, will compare dates loosely since you often compare existing documents (dates in ISO string) with documents that just have been scraped where dates are Dates.

Kind: instance method of Document

fs

Manifest is provided differently in developement that in production.

In production, the manifest has been "merged" via Webpack via the DefinePlugin
In development/test, we simply read the manifest from the fs

Kind: global constant

LOGIN_FAILED : `string`

The konnector could not login

Kind: global constant

NOT_EXISTING_DIRECTORY : `string`

The folder specified as folder_to_save does not exist (checked by BaseKonnector)

Kind: global constant

VENDOR_DOWN : `string`

The vendor's website is down

Kind: global constant

USER_ACTION_NEEDED : `string`

There was an unexpected error, please take a look at the logs to know what happened

Kind: global constant

FILE_DOWNLOAD_FAILED : `string`

There was a problem while downloading a file

Kind: global constant

SAVE_FILE_FAILED : `string`

There was a problem while saving a file

Kind: global constant

DISK_QUOTA_EXCEEDED : `string`

Could not save a file to the cozy because of disk quota exceeded

Kind: global constant

CHALLENGE_ASKED : `string`

It seems that the website requires a second authentification factor that we don’t support yet.

Kind: global constant

LOGIN_FAILED_TOO_MANY_ATTEMPTS : `string`

Temporarily blocked

Kind: global constant

USER_ACTION_NEEDED_OAUTH_OUTDATED : `string`

Access refresh required

Kind: global constant

USER_ACTION_NEEDED_ACCOUNT_REMOVED : `string`

Unavailable account

Kind: global constant

USER_ACTION_NEEDED_CHANGE_PASSWORD : `string`

Unavailable account

Kind: global constant

USER_ACTION_NEEDED_PERMISSIONS_CHANGED : `string`

Password update required

Kind: global constant

USER_ACTION_NEEDED_CGU_FORM : `string`

The user needs to accept a CGU form before accessing the rest of the website

Kind: global constant

CAPTCHA_RESOLUTION_FAILED : `string`

solveCaptcha failed to solve the captcha

Kind: global constant

LOGIN_FAILED_NEEDS_SECRET : `string`

Additionnal information is needed to check your login details

Kind: global constant

MAINTENANCE : `string`

remote website seems to be unavailable

Kind: global constant

TERMS_VERSION_MISMATCH : `string`

User needs to accept new terms

Kind: global constant

UNKNOWN_ERROR : `string`

unkown error

Kind: global constant

UNKNOWN_ERROR_PARTIAL_SYNC : `string`

The synchronization is complete but some elements may be missing

Kind: global constant

USER_ACTION_NEEDED_SCA_REQUIRED : `string`

Renewal of authentication required

Kind: global constant

USER_ACTION_NEEDED_TWOFA_EXPIRED : `string`

Authentication renewal required

Kind: global constant

USER_ACTION_NEEDED_WEBAUTH_REQUIRED : `string`

Authentication on vendor website required

Kind: global constant

USER_ACTION_NEEDED_WRONG_TWOFA_CODE : `string`

Incorrect strong authentication code

Kind: global constant

VENDOR_DOWN_BANK_DOWN : `string`

Unavailable bank website

Kind: global constant

VENDOR_DOWN_LINXO_DOWN : `string`

Unavailable bank website

Kind: global constant

attachProcessEventHandlers(prcs) ⇒ `function`

Attach event handlers to catch uncaught exceptions/rejections and signals. Log them as critical and exit the process accordingly. If the cleanup function has not been called, calling again the function is a no-op.

Kind: global function
Returns: function - When called, removes the signal handlers

Param	Type	Description
prcs	`object`	Process object, default to current process

mkSpec()

Declarative scraping.

Describe your items attributes and where to find/parse them instead of imperatively building them.

Heavily inspired by artoo scraping method.

Kind: global function

scrape($, specs, [childSelector]) ⇒ `object` | `Array`

Scrape a cheerio object for properties

Kind: global function
Returns: object | Array - - Item(s) scraped

Param	Type	Description
$	`object`	Cheerio node which will be scraped
specs	`object` \| `string`	Options object describing what you want to scrape
[childSelector]	`string`	If passed, scrape will return an array of items

Example
scrape can be used to declaratively extract data :

For one object :

const item = scrape($('#item'), {
  title: '.title',
  content: '.content'
})

For a list of objects :

const items = scrape($('#content'), {
  title: '.title',
  content: '.content'
}, '.item')

For more power, you can use objects for each retriever :

const items = scrape($('#content'), {
  title: '.title',
  content: '.content',
  link: {
    sel: 'a',
    attr: 'href'
  },
}, '.item')

Here the href attribute of the a inside .items would have been put into the link attribute of the items returned by scrape.

Available options :

sel: the CSS selector used to target the HTML node from which data will be scraped
attr: the HTML attribute from which to extract data
parse: function applied to the value extracted ({ sel: '.price', parse: parseAmount })
fn: if you need something more complicated than attr, you can use this function, it receives the complete DOM node. { sel: '.person', fn: $node => $node.attr('data-name') + $node.attr('data-firstname') }

⚠ Permissions

Please note that some classes require some permissions:

io.cozy.accounts for the BaseKonnector class (GET only)
io.cozy.files to save files
io.cozy.bills to save bills
io.cozy.bank.operations for linkBankOperations

Files

api.md

Latest commit

History

api.md

File metadata and controls

API

Modules

Classes

Constants

Functions

CozyBrowser

defaultOptions ⇒ object ⏏

defaultOptions~addListeners()

addData

addData() ⏏

cozyClient

cozyClient ⏏

hydrateAndFilter

hydrateAndFilter() ⏏

hydrateAndFilter~suitableCall()

linkBankOperations

linkBankOperations() ⏏

mkdirp

mkdirp ⏏

normalizeFilename

normalizeFilename() ⏏

saveBills

saveBills() ⏏

saveFiles

saveFiles(entries, fields, options, [contract]) ⏏

saveIdentity

saveIdentity() ⏏

signin

signin() ⏏

solveCaptcha

solveCaptcha() ⏏

updateOrCreate

updateOrCreate() ⏏

utils

utils~fetchAll()

utils~queryAll()

utils~findDuplicates()

utils~batchUpdateAttributes()

utils~batchDelete()

utils~getPdfText()

utils~formatDate()

categorization

categorization~createCategorizer(options) ⇒ Object

categorization~categorize(transactions, options) ⇒ Array.<object>

categorization~CreateCategorizerOptions

BaseKonnector

new BaseKonnector(fetch)

baseKonnector.run()

baseKonnector.main() ⇒ Promise

baseKonnector.end()

baseKonnector.fail()

baseKonnector.readPayload() ⇒ Object | null

baseKonnector.initAttributes()

baseKonnector.saveAccountData(data, options) ⇒ Promise

baseKonnector.getAccountData() ⇒ object

baseKonnector.updateAccountAttributes()

baseKonnector.setTwoFAState(options)

baseKonnector.resetTwoFAState()

baseKonnector.waitForTwoFaCode(options) ⇒ Promise

baseKonnector.notifySuccessfulLogin()

baseKonnector.deactivateAutoSuccessfulLogin()

baseKonnector.saveBills() ⇒ Promise

baseKonnector.saveFiles() ⇒ Promise

baseKonnector.updateOrCreate() ⇒ Promise

baseKonnector.saveIdentity() ⇒ Promise

baseKonnector.signin() ⇒ Promise

baseKonnector.terminate(err)

baseKonnector.getCozyMetadata(data)

CookieKonnector

new CookieKonnector(requestFactoryOptions)

cookieKonnector.initAttributes() ⇒ Promise

cookieKonnector.end()

cookieKonnector.requestFactory(options) ⇒ object

cookieKonnector.resetSession() ⇒ Promise

defaultOptions ⇒ `object` ⏏

categorization~createCategorizer(options) ⇒ `Object`

categorization~categorize(transactions, options) ⇒ `Array.<object>`

baseKonnector.main() ⇒ `Promise`

baseKonnector.readPayload() ⇒ `Object` | `null`

baseKonnector.saveAccountData(data, options) ⇒ `Promise`

baseKonnector.getAccountData() ⇒ `object`

baseKonnector.waitForTwoFaCode(options) ⇒ `Promise`

baseKonnector.saveBills() ⇒ `Promise`

baseKonnector.saveFiles() ⇒ `Promise`

baseKonnector.updateOrCreate() ⇒ `Promise`

baseKonnector.saveIdentity() ⇒ `Promise`

baseKonnector.signin() ⇒ `Promise`

cookieKonnector.initAttributes() ⇒ `Promise`

cookieKonnector.requestFactory(options) ⇒ `object`

cookieKonnector.resetSession() ⇒ `Promise`

cookieKonnector.initSession() ⇒ `Promise`

cookieKonnector.saveSession() ⇒ `Promise`

cookieKonnector.signin() ⇒ `Promise`

cookieKonnector.saveFiles() ⇒ `Promise`

cookieKonnector.saveBills() ⇒ `Promise`

LOGIN_FAILED : `string`

NOT_EXISTING_DIRECTORY : `string`

VENDOR_DOWN : `string`

USER_ACTION_NEEDED : `string`

FILE_DOWNLOAD_FAILED : `string`

SAVE_FILE_FAILED : `string`

DISK_QUOTA_EXCEEDED : `string`

CHALLENGE_ASKED : `string`

LOGIN_FAILED_TOO_MANY_ATTEMPTS : `string`

USER_ACTION_NEEDED_OAUTH_OUTDATED : `string`

USER_ACTION_NEEDED_ACCOUNT_REMOVED : `string`

USER_ACTION_NEEDED_CHANGE_PASSWORD : `string`

USER_ACTION_NEEDED_PERMISSIONS_CHANGED : `string`

USER_ACTION_NEEDED_CGU_FORM : `string`

CAPTCHA_RESOLUTION_FAILED : `string`

LOGIN_FAILED_NEEDS_SECRET : `string`

MAINTENANCE : `string`

TERMS_VERSION_MISMATCH : `string`

UNKNOWN_ERROR : `string`

UNKNOWN_ERROR_PARTIAL_SYNC : `string`

USER_ACTION_NEEDED_SCA_REQUIRED : `string`

USER_ACTION_NEEDED_TWOFA_EXPIRED : `string`

USER_ACTION_NEEDED_WEBAUTH_REQUIRED : `string`

USER_ACTION_NEEDED_WRONG_TWOFA_CODE : `string`

VENDOR_DOWN_BANK_DOWN : `string`

VENDOR_DOWN_LINXO_DOWN : `string`

attachProcessEventHandlers(prcs) ⇒ `function`

scrape($, specs, [childSelector]) ⇒ `object` | `Array`