Installing the Filesystem Connector

Cloud Customers

The Filesystem connector can be installed in any servlet container behind your firewall where it can connect to your back end system in order to read/write data.

1. If you use the Live Forms Cloud hosted server,  you must ensure that the connector is accessible from the frevvo servers. This is done by opening up one of the following ports in your firewall:
   
   
   • HTTP Ports: 80 or 19080

   • HTTPs Ports: 443 or 19443
     
     

   We recommend that you use https and not http. Https is a secure channel.

2. Customers that use firewall white-listing, should make sure the frevvo cloud outgoing server IP 54.86.85.105 (host outgoing.frevvo.com) is added to the list.
3. If you want to install the Filesystem Connector in the frevvo In-House tomcat bundle, follow these easy steps: (You don't have to use the  application from this bundle).
   
   1. Review the Supported Platforms list.
   2. Download and install the certified version of the Java Development Kit (JDK) or Java Runtime Environment (JRE) and set the JAVA_HOME environment variable as described here 
      
   3. Download the frevvo tomcat bundle
      
   4. Unzip the frevvo zip file into a location of your choice: e.g. c:\frevvo.   We'll refer to this directory below as <frevvo-home>.
      
   5. Install the Filesystem connector inside that same tomcat. 
      
      1. Download the filesystem zipflile 
      2. Extract the filesystem.war to <frevvo-home>\tomcat\webapps directory.
      3. Delete <frevvo-home>\tomcat\webapps\frevvo.war
         
      4. Configure the Filesystem Connector

4. Start by clicking on the startup.(bat,sh) file for your operating system in the <frevvo-home> directory. It is also possible to configure Live Forms as a Windows service
   

In-house Customers with an existing tomcat bundle installation

The Filesystem connector can be installed in any servlet container behind your firewall where it can connect to your back end system in order to read/write data. These instructions assume you have an in-house installation of  up and running and you are running and the Filesystem Connector in the same tomcat.

To install the Filesystem connector in the same tomcat as :

1. Stop .
2. Download the filesystem connector zipfile
3. Extract filesystem.war to your <frevvo-home>\tomcat\webapps directory.
4. Configure the Filesystem Connector
5. Start
   

Upgrading from a previous version

To upgrade a previous version of the Filesystem Connector to the latest version in your existing Tomcat bundle installation:

1. Stop

2. Download the latest version of the Filesystem Connector

3. Extract the filesystem.war to your <frevvo-home>\tomcat\webapps directory.

4. Verify that the Filesystem Connector configuration is correct for the new version. You will need to convert the context parameters in the the <frevvo-home>\tomcat\conf\catalina\localhost\filesystem.xml file from your previous installation to properties in the <frevvo-home>\tomcat\conf\frevvo-config.properties file.

    Click here to see how to convert the parameters from the filesystem.xml file to properties in the frevvo-config.properties file

   1. Let's say your filesytem.xml file from your previous version looks like this:
      
      

      <Context unpackWAR="false" path="/filesystem" swallowOutput="true" unloadDelay="40000">
      <Parameter name="spring.main.banner-mode" value="OFF" override="false"/>
      <Parameter name="logging.file" value="${catalina.base}/logs/file-connector.%d{yyyy-MM-dd}.log" override="false"/>
      <Parameter name="connector.file.root.dir" value="C:\" override="false"/>
      <Parameter name="connector.file.name.keep" value="true" override="false"/>
      <Parameter name="connector.file.ignore.attachments" value="false" override="false"/>
      <Parameter name="connector.file.descriptor.suffix" value=".xml" override="false"/>
      <JarScanner scanClassPath="false"/>
      </Context>

   2. Edit the <frevvo-home>\tomcat\conf\frevvo-config.properties file
   3. The Context unpackWar, spring.main.banner-mode, logging.file and JarScanner scanClassPath parameters from the filesystem.xml file are not needed.
   4. Convert the remaining parameters to properties. You can add them to the existing File Connector settings section or make a new section with a comment as shown in the example below
      
      1. Copy the parameter name without the surrounding quotes into the frevvo-config.properties file. This is the name of the property.
      2. Add an equal sign after the property name
      3. Copy the value without the surrounding quotes and add it AFTER the equal sign.
      4. Repeat for all the configured parameters
      5. Save the frevv-config.properties file

   The frevvo-config.properties file should look similar to the one shown below after the conversion:

   • In a properties file, a backslash is considered as a continuation character when splitting across lines. Escape it with another backslash so it is not treated as such. i.e. C:\\ or use the forward slash i.e. c:/ when specifying the connector.file.root.dir property.

   • You may notice the following properties for the Filesystem connector listed in the frevvo-config.properties file by default. In previous versions of these properties were configured as parameters in the <frevvo-home>\tomcat\conf\catalina\localhost\frevvo.xml file. If you modified the values for these properties in the frevvo.xml of your previous installation, copy those values into the default properties listed in the frevvo-config.properties file.

   • Uncomment the properties by deleting the # that precedes them if you are using them.

     # File Connector settings
     #frevvo.filesystem.connector.url=http://localhost:8082/filesystem
     #frevvo.filesystem.connector.metadata.prefix=md.

   The first statement, preceded by the #, in the example shown below is a comment for informational purposes.

   #Filesystem Connector properties
   connector.file.root.dir=C:/
   connector.file.name.keep=true 
   connector.file.ignore.attachmentse=false
   connector.file.descriptor.suffix=.xml

   5. Start

5. Proceed to Connect Forms/Flows with the Save to FileSystem wizard to continue.

Configuring the Filesystem Connector

The Filesystem Connector is configured using properties in the <frevvo-home>\tomcat\conf\frevvo-config.properties file.

Follow these steps to configure the Filesystem connector:

1. You may notice the following properties listed in the File Connector settings section of the frevvo-config.properties file.

   1. the frevvo.filesystem.connector.url - The location of the Filesystem connector is set for the installation with this property. It defaults to http://localhost:8082/filesystem. The <localhost:8082> in the default URL can be changed to the <server:port> where you installed the Filesystem connector in your environment.

   2. the frevvo.filesystem.connector.metadata.prefix - this property is needed to retrieve field mapping when generating a metadata file. It defaults to md followed by a dot. In most situations, there is no need to change this value.

   3. Remember to uncomment the properties by deleting the # that precedes them if you are using them.

      # File Connector settings
      #frevvo.filesystem.connector.url=http://localhost:8082/filesystem
      #frevvo.filesystem.connector.metadata.prefix=md.

2. Add these properties to the <frevvo-home>\tomcat\conf\frevvo-config.properties file. You can add them to the existing File Connector settings section or make a new section with a comment as shown in the example below. Refer to the table for a description of the configurable properties:

   Configuration Parameter	Description
   connector.file.root.dir	This is the location in your file system where the submission docs will be saved.  Set the submissions directory field in the Save to Filesystem wizard as a relative path from the root directory specified by the connector.file.root.dir in the connector deployment configuration. It's value can contain one or more control template strings. For example, if the configuration file contains C:\ for the connector.file.root.dir parameter, enter the name of the directory where you want to save the submissions in the Submissions field in the Save to Filesystem wizard. This will save the submissions in C:\<DIRECTORY NAME>. If not explicitly configured, it defaults to the user home directory. Change the value to reflect the location for your file system.
   connector.file.name.keep	This parameter determines whether the original files names will be retained. If this is set to true, then you either have to make sure that all your files have unique names or else save the documents of each submission under a separate uniquely named folder. If this parameter is not included, or is set to false, submission files will be named with a generated identifier as the prefix suffixed by '_' and a sequence number to make the file name unique. The original filename extension is retained. This is the default behavior. Ex: e454fa08-9a6e-4c54-9743-18f4ba0ad0b4_1.xml instead of form.xml.
   connector.file.dir.submission	This parameter allows you to create a unique new sub-folder for documents of every submission if true.
   connector.file.ignore.attachments	This parameter determines whether attachments will be saved or not. If the value is false, attachments will be saved. Change the value to true and attachments will not be saved. The submission XML does not store the names of uploaded attachments, instead it points to unique ID that is assigned to each attachment when that submission in created in . This is as-designed.
   connector.file.descriptor.suffix	The metadata descriptor filename is a generic GUID filename, followed by a configurable suffix. The default suffix is '.xml'. Use this parameter to change the suffix.The suffix can make identifying the metadata file easier, particularly if you save the files from all submissions into the same folder. Ex: 432e0d1f-d084-4cd4-8bbe-50f6e9bd5ee6_desc.txt

   Here is an example of the Filesystem Connector properties added to the File Connector settings section of the frevvo-config.properties file. When executed, submissions will be:

   • Saved on the local drive in a directory specified in the wizard that is located under the root directory configured here. Ex: C:\Submissions.
   • Individual file names will be retained.
   • Attachments in the submission will be saved.

   • Add the suffix "_desc.txt" to the metadata file for each submission.
     

     In a properties file, a backslash is considered as a continuation character when splitting across lines. Escape it with another backslash i.e. C:\\ or use the forward slash i.e. C:/ when specifying the connector.file.root.dir property.

     # File Connector settings
     #frevvo.filesystem.connector.url=http://localhost:8082/filesystem
     frevvo.filesystem.connector.metadata.prefix=md.
     connector.file.root.dir=C:/
     connector.file.name.keep=true 
     connector.file.ignore.attachments=false
     connector.file.descriptor.suffix=_desc.txt
     

     
     

3. Start . If you are running v7.0+, remember to start the Insight server first.

4. Make sure the directory where you plan to save the submissions is created and and has the appropriate access for the OS you are using. For example:

   1. If you are using Windows, you can map a drive to a server directory, or you could use the UNC path i.e. //host-name

   2. If you are using a Unix operating system you could mount a remote filesystem as a directory and use it as your destination directory

      To ensure interoperability across file systems, it is recommended that you follow a good naming standard such as POSIX when creating files and folders. For example, folder names can't contain slashes so a folder named '1/2/15' would not be valid.

5. Browse http://<your server:port>/file/health (replace <your server:port> with your frevvo server:port) to check the status of the connector. If the connector is up and running, you will see { "status" : "UP" }

Connect Forms/Flows with the Save to File System wizard

Saving files to a local or remote location is easily done with the Save to File System wizard if generating metadata is not required. Let's begin with a very simple example. Here are the requirements:

1. You have created a small form that has a Name, Address, City, State and Zip Code fields. A mapped W - 4 PDF is included in the form. The Save PDF property is selected and the PDF Name property is configured with the template {Name}.

2. You want to store the submission files for this form in a directory on your local machine named Submissions. 
3. You want to create a sub directory for each submission under the Submissions directory. The name of the sub directory is created from the value entered into the Name field on the form
4. You want to save the form/flow PDF, any mapped PDFs associated with the form, an xml file and any attachments uploaded to the form in the subdirectory. 
5. Follow these steps to fulfill the requirements by using the doc action Save to Filesystem wizard
   1. Login as a designer.

   2. Edit your form/flow in the appropriate designer.

   3. Click on the what happens to my data? icon.
      
      
      
      

   4. Click Save to FileSystem to display the wizard.
      
      

6. To fulfill the requirements listed above, we only need to fill in the URL, Submission Directory and File sections of the wizard. Here is an explanation of the fields:
    

   1. The URL: this is the URL to the location where you installed the Filesystem Connector.

      1. In-house: The location of the Filesystem connector is set for the installation using the frevvo.filesystem.connector.url parameter in the <frevvo-home>\tomcat\conf\frevvo-config.properties file. It defaults to http://localhost:8082/filesystem in the tomcat build. The <localhost:8082> in the default URL can be changed to the <server:port> where you installed the Filesystem connector in your environment. This is a required field. The value entered here overrides the value in the frevvo-config.properties file.

      2. Cloud customers will see the placeholder http(s)://host:port/filesystem/submission - enter the https url to the location where you installed your  Filesystem Connector.
         
         
         

          

   2. Submissions Directory - Enter a relative path from the root directory specified by the connector.file.root.dir parameter in the configuration file created above. Control templates are allowed in this field. You can use a control template to store submissions in subdirectories. The template, {Name}, will populate the Template field. Copy the template from the Template field and add it to the Submissions Directory path after the forward or backward slash. Note the forward or backward slash can be used to create subfolders.
      
      Examples
   •  if the value of the connector.file.root.dir parameter in the configuration file is "c:\" then entering Submissions in the Submission Directory field in the wizard saves the files in the local directory c:\Submissions.
      
   1. File Section - One or more types of submission content can be specified by checking the Send Snapshot, Files, Gen Forms and Data checkboxes:
      1. Select the snapshot format to be sent by selecting a choice form the Send Snapshot dropdown. The format choices are PDF, PNG or JPEG.
      2. Check the Files checkbox to send any file attachments uploaded to the form.
      3. Check the Gen Forms checkbox to send the mapped W - 4 PDF.
      4. Check the Data checkbox to send the form XML document for the form.
         
         
         
         
7. Click the Submit button.
8. Click the save and exit icon to save your changes.
9. Create some submissions by clicking the Test button, filling in the form and clicking submit.
10. The results of 3 submissions are shown in the image.
    
    

Generating Metadata

The Filesystem Connector saves the submission form xml, attachments and PDFs in the configured location. In addition, a metadata file, which describes the data, is created. This is helpful if your document management system has the capability to import files.

Generating metadata in xml format is the most common use case.

Follow these steps to connect your form/flow to the Filesystem Connector:

1. Create a descriptor file named file-descriptor.ftl in your <frevvo-home>\tomcat\lib directory. The file MUST be named file-descriptor.ftl.
2. Use the Save to File System wizard to generate a map to provide the submission data to the connector 
   
   

Create the Metadata Descriptor File Template

The metadata descriptor file template generates a metadata file for every submission using the map that is generated by the wizard and the template. This file will vary depending on the ECM that you are using and your specific requirements.

1. If you are using the tomcat bundle, it should be located in the <frevvo-home>\tomcat\lib directory.

2. The content of the file-descriptor.ftl file uses freemarker template language v2.3.23 syntax.

Below are examples of the filesystem descriptor files that integrate with the DocuWare Management System, DocuShare Management System and ImageNow Imaging and Document Management Systems.

File Descriptor File Examples

Let's take a look at the file-descriptor.ftl file for DocuShare. In general, freemarker syntax is identified by the <# symbols.

• In the file, #compress, #escape, #function, #return, etc... are freemarker instructions
• The characters in this section of the file will be included in the metadata output file interspersed with freemarker instructions.
  • The submission.file.name is a special string, that adds a line in the xml file with a full path to the location where attachments, form/mapped PDFs and the form.xml are being saved for the submission.
• The last two lines in the file, </#escape and </#compress are the closing tags for the instructions specified at the top of the file.

These examples are offered as a guide to help you create your filesystem descriptor file. Even if you are working with one of the ECM systems represented by the examples below, you will need to customize the descriptor file to work for you.

<#compress>
<#escape x as x?xml>
<#function isSeq val>
   <#return (val?starts_with("[") && val?ends_with("]"))>
</#function>
<#function toSeq val>
   <#if isSeq(val)>
   <#attempt><#return val?eval><#recover><#return val></#attempt>
   <#else><#return val>
   </#if>
</#function>
<?xml version="1.0" encoding="UTF-8"?>
<Records>
  <Record>
    <Field id="LF__User" length="-1" type="string">
      <Value>administrator</Value>
    </Field>
  <#list params?keys as key>
  <#assign values = (params[key]?size > 1) || isSeq(params[key][0])>
  <#assign doc = key?switch('submission.file.name',true,false)>
    <Field id="${doc?then('LF__Document',key)}" length="-1" type="${doc?then('document','string')}">
      <#if values><Values></#if>
      <#list params[key] as value><#assign attribs = toSeq(value)>
      <#if attribs?is_enumerable>
         <#list attribs as attrib><Value>${attrib}</Value></#list>
      <#else><Value>${value}</Value>
      </#if>
      </#list>
      <#if values></Values></#if>
    </Field>
  </#list>
  </Record>
</Records>
</#escape>
</#compress>

<#compress>
<#escape x as x?xml>
<?xml version="1.0" encoding="UTF-8"?>
<ControlStatements xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://dev.docuware.com/Jobs/Control">
  <Document>
     <#list params['submission.file.name'] as value><InsertFile path='${value}' /></#list>
  </Document>
  <Page>
  <#list params?keys as name>
  <#if name != "submission.file.name">
    <#list params[name] as value>
    <#assign attribs = value?eval>
    <Field dbName="${name}" <#list attribs?keys as key>${key}="${attribs[key]}" </#list>/>
    </#list>
  </#if>
  </#list>
  </Page>
</ControlStatements>
</#escape>
</#compress>

<#compress>
<#escape x as x?xml>
<?xml version="1.0" encoding="UTF-8"?>
<Document>
  <Key name="filename" type="filename">
     <#list params['submission.file.name'] as value><Value>${value}</Value></#list>
  </Key>
  <#list params?keys as name>
  <#if name != "submission.file.name">
    <#list params[name] as value>
    <#assign attribs = value?eval>
    <Key name="${name}" <#list attribs?keys as key>type="${key}"><Value>${attribs[key]}</Value></#list></Key>
    </#list>
  </#if>
  </#list>
</Document>
</#escape>
</#compress>

Generating Metadata using the Save to File System wizard

Once the file-descriptor.ftl file is in place, the wizard must be configured to specify the Name-value pairs that will be used to generate the metadata file. This is done in the Metadata section of the wizard. The Name and Value columns are required. The Metadata Prefix is used to distinguish submission data fields from metadata fields. The default value of the prefix is md. The Name or Group value is prefixed with the metadata prefix when the doc action is executed.

1. The Name column is to used to represent a unique entry on its own or within a Group. The following entry will cause a md.name=Tray Jenkins parameter to be sent to the connector.

   Name	Value
   name	Tray Jenkins

2. The Group column is used to group name-value pairs. To represent a metadata entry with attributes you would use the Group column to indicate the entry name. The following entires will cause a md.EMPLOYMENTDATE={"type":"Date","value":"2014-04-03","culture":"en-US","format":"yyyy-MM-dd"} parameter to be sent to the connector.

   	Value	Group
   type	Date	EMPLOYMENTDATE
   value	2014-04-03	EMPLOYMENTDATE
   culture	en-US	EMPLOYMENTDATE
   format	yyyy-MM-dd	EMPLOYMENTDATE

3. The Value column can contain one or more control template strings

4. The Controls dropdown contains the labels of all controls in the form or flow. To create a template string, select a control. This causes the adjacent Template field to be populated with the correct template string. This can then be copied into the fields that support template resolution.

5. To include values from Repeating controls in your form/flow, the value should be a template string that resolves to multiple values. The recommended approach is to create a template that results in a JSON array representation such as. ["{","|controlname}"].
   
   

Filesystem Connector Example:

Let's say you have a small form with the following controls: First Name, Last Name, Address, Date of Birth, a Signature control and a repeating control named State. Using the DocuShare file-descriptor.ftl file and setting up the The Save to Filesystem wizard as shown in the above image results in these files the submission directory.

The generated metadata file contains the following:

<?xml version="1.0" encoding="UTF-8"?>
<Records>
<Record>
<Field id="LF__User" length="-1" type="string">
<Value>administrator</Value>
</Field>
<Field id="my_name" length="-1" type="string">
<Value>{&quot;First Name&quot;:&quot;Arnold&quot;,&quot;Last Name&quot;:&quot;Smith&quot;}</Value>
</Field>
<Field id="Address" length="-1" type="string">
<Value>34 Some Street
New City</Value>
</Field>
<Field id="EMPLOYMENTDATE" length="-1" type="string">
<Value>{&quot;culture&quot;:&quot;en-US&quot;,&quot;format&quot;:&quot;yyyy-MM-dd&quot;,&quot;type&quot;:&quot;3/31/1954&quot;,&quot;value&quot;:&quot;3/31/1954&quot;}</Value>
</Field>
<Field id="State" length="-1" type="string">
<Values>
<Value>NY</Value><Value>MN</Value>
</Values>
</Field>
<Field id="LF__Document" length="-1" type="document">
<Values>
<Value>c:\Submissions\Arnold Smith\form.xml</Value>
<Value>c:\Submissions\Arnold Smith\Letter of Recommendation 1.docx</Value>
<Value>c:\Submissions\Arnold Smith\Arnold Smith.pdf</Value>
<Value>c:\Submissions\Arnold Smith\W - 4 for Smith</Value>
</Values>
</Field>
</Record>
</Records>

Troubleshooting

The Filesystem Connector generates it's own logfile. Typically, the log is first place to look for information when an issue is encountered. Information about the logfile and some common issues with solutions are listed below.

Filesystem Connector Logfile

If you are using the frevvo tomcat bundle, the Filesystem Connector log is located in <frevvo-home>\tomcat\logs. The logfile is named filesystem-connector.YYYY-MM-DD.log. Generating separate logfiles make it much easier to search for errors when troubleshooting connector issues. The connector is designed to use the frevvo-config.properties file.

The Filesystem connector logfile defaults to INFO loglevel. To configure DEBUG level logging for the Filesystem connector log:

1. Stop
2. Create/edit a frevvo-config.properties file in <frevvo-home>\tomcat\conf
3. Add the logging.level.com.frevvo.connectors.file=DEBUG property to the file. 
4. Save the file
5. Restart
   
   

logging.level.com.frevvo.connector.file=DEBUG

Doc Post Failure

If the destination for saving submissions that you have specified cannot be reached, all users with the tenant admin role or a flow admin (if configured) will receive an Doc Post Failure notification email reporting an HTTP Status error - 404.

Check the connector.file.root.dir parameter in the file.xml file, the URL in the POST data to your web application wizard or the Save to File System wizard screen depending on the wizard you are using.

Determine the version of the Filesystem Connector you are using:

To determine the version of the Filesystem connector that you are using:

1. Browse http://<your server:port>/filesystem/info  - the version information is returned as shown in the image.