Startups and Product Verification Testing


The purpose of product verification testing is to verify the product meets its’ design goals, and is included in any set of best practices. But what really is being verified?

  • Compliance to undocumented or informal assumptions of user expectations?
  • Compliance to documented external requirements, potentially including regulatory requirements?
  • Compliance to internal design goals, possibly including a code standard, static code analysis, and unit testing?
  • Confirm buttons can be pressed in any sequence and nothing bad happens?
  • Combinations of the above?

Formal complete verification testing is a valiant goal, but difficulties can be encountered in practice, particularly when funding is limited. A good formal test phase can cost as much in calendar days and labor $$’s as the design phase! A startup may not have the funding and runway for a good test phase, and could even be prejudiced against test by overconfident designers and managers. 

Unless verification test is represented on the project team from its onset, test resources will usually get addressed late in the development phase – once there is something to test. This means developers will first need to spend time onboarding the test resource to a suitable level familiarity with the product and the codebase, and more time during the test process to clarify ambiguous behavior and investigate test failures. A more viable strategy can be to leverage the design team’s intimate knowledge of their system in the test effort.

I like to think of product verification testing as a stool with three legs, one leg is automated regression testing, one leg is feature-specific testing, and the final leg is ad hoc “try to break it” attempts.

  • Use an automated test to catch obvious blunders with minimum effort, and run the test on every release candidate, every night, or even after every commit to the code repo. Creating the test will unavoidably take time from the development team, but often the time is not significantly more than that needed to on-board a developer-level test resource, explain requirements and validate the created test. 
  • Use feature-specific tests for verifying areas of concern where there is high risk, or where there is high consequence (such as regulatory compliance). Existing areas of concern by team members can be identified and explored in team whiteboard sessions (e.g. a concern the system may lockup if a user is interacting with the device when the internet connection is lost), and fault tree and/or failure-mode analysis diagrams can be created for further resolution and clarity. It may be practical to involve a temporary resource at this point to off-load the development team (e.g. an engineering or computer science co-op or summer student) and the test phase will have a higher chance of success, instead of muddling about for a month or two due to lack of support and produce no tangible results.
  • Use ad hoc “break-it” testing to find issues that would never be contemplated by the design team. The tester should not be familiar with the design internals, but should preferably be at least familiar with the product domain. As such, it can often be off-loaded to your favorite student again. However, be cautious of expecting useful test feedback from beta testers. Despite best intentions, beta testers are typically not motivated to spent effort in areas of functionality not of direct interest, or to document found issues to the degree needed by the development team.

Separation of design and test, documented requirements, formal verification, etc. have become best practices for a reason, but may not be practical or feasible for a startup chasing their first product. If the startup is successful at creating a revenue stream to protect, it will be able to adopt or refine its practices. If the startup fails due to having built the wrong product, in hindsight clearly resources spent on verification would have been better put towards market intelligence and alpha testing a proof-of-concept.

However, of course this is all IMHO and YMMV….

Top 10 Enterprise Open Source Software

This isn’t really a Top 10. There are only seven, but also because these software applications are largely independent and a solution could use several. But now that you’re here… 😉

Open source is about a community rallying around a common software solution to a shared problem, and volunteering effort (e.g. writing code, writing web pages to help others, moderating or helping others in a forum, testing code, fixing bugs…) in return for the benefits they derive. There may be a primary commercial entity involved that monetizes some aspect of the software, or not. Every community is different, because the itch they scratch is different.

That being said, these applications should be of interest to any technically-oriented business process manager. They solve general problems, such as financial transactions, inventory management, controlled document sharing, issue reporting and corrective action processes, effectively progressing leads to orders, managing production and supply chain, scheduling manufacturing builds, project reporting, etc. They each have parallels in commercial proprietary software, which gives a clearly identifiable cost benefit.

  • ERPNext is a fully featured web-based ERP system and leader in the era of postmodern ERP. As powerful as it is easy to use, ERPNext can be used stand-alone or to enhance the capabilities of an existing system. Commercial support is provided by ERPNext sponsor Frappe Technologies, who offer a variety of services, and by a strong network of independent consultants.
  • Nextcloud is the most deployed on-premises file share and collaboration platform for tens of millions of users at thousands of organizations across the globe. Commercial support is available from Nextcloud sponsor Nextcloud GmbH, who also provide certified compliance, and by a strong network of independent consultants. 
  • MantisBT was developed as a software bug tracker, but it also makes an effective enterprise issue management system. Immediate benefits are managing projects more effectively and collaborating efficiently over issues. Effective support is provided by the MantisBT community.
  • SuiteCRM is fully-featured Customer Relationship Management software. Starting life in 2004 as open source SugarCRM, SuiteCRM was created in 2015 in order to continue as an open source project. Commercial support is available from the SuiteCRM sponsor SalesAgility, and the thriving ecosystem includes many professional support options and development partners.
  • TimeTracker is an easy-to-host easy-to-use timesheet system for an entreprise starting to capture time-on-task for project management purposes. It can also generate invoices for consultants and contractors, notify workers and managers of incomplete assignment, and lock time entry after approval. TimeTracker is supported by its creator and primary developer, and by an active community.
  • ProjeQtOr is open source enterprise project management software. The name is an acronym for Quality based Project Organizer, to reference a strong quality management philosophy which includes indicators, alerts and defined workflow. ProjeQtOr will be of particular interest to those familiar with commercial portfolio project management software. Support is available from the developer through a variety of commercial services, and also from a strong user community. 
  • webERP is the original web-based open-source ERP software for web-based small business accounting. Founded in 2001 by a New Zealand accountant, webERP has proven to be especially suitable for small businesses with slow internet connections, and is supported by a network of independent consultants and contractors.

These programs are available for guided demonstration on dalescott.net. Please request a demo using the contact form.

Selling my Soul?

I received my monthly activity report for dalescott.net traffic from Google today, a thought-provoking top-level summary provided at “no cost” (although some will say it’s already cost me my soul ;-)).

This is technology research for me. Big enterprise expends effort where a return is expected. I follow their lead looking for the concepts that will provide immediate value to smaller businesses, and that can be implemented at a fraction of the cost through participation in Open Source ecosystems. 

Building a Tryton Server on FreeBSD

This article is part of a series on the Tryton framework, to complement the Maestro project on GitHub. This article describes installing the Tryton server (trytond) on FreeBSD.

Caution – this is a Work in Progress  updating original 2014-05-24 post for Tryton 5.2 on FreeBSD 12.0 using Virtualenv.

Install FreeBSD

Boot vm from boot CD/DVD and follow the standard install procedure, including creating an admin user and including it in the wheel group.

I always configure remote access using ssh keys only for for security, Copy the public ssh key for the admin user to ~/.ssh, edit /etc/ssh/sshd_config to add “AllowUsers adminusername” and restart sshd.

Check for and install available FreeBSD OS updates, and install the pkg packaging system port..

# freebsd-update fetch
# freebsd-update install
# pkg update   

You can check the installed packages for reported vulnerabilities (-F is required for initial use only to download a new vulnerability database).

# pkg audit -F

Install PostgreSQL

The files belonging to the Postgresql database system will be owned by user “postgres”, who must also own the server process.

# pkg install postgresql93-server-9.3.4
# echo "postgresql_enable=YES" >> /etc/rc.conf
# /usr/local/etc/rc.d/postgresql initdb
# /usr/local/etc/rc.d/postgresql start

The default configuration file (/var/db/postgres/data96/postgresql.conf is acceptable and does not need editing. Only localhost will have access to PostgreSQL, which is adequate as Tryton will be served from the same server. 

Add a PostgreSQL “tryton” super-user.

dale@casper:~ % sudo su postgres
$ createuser -sdrP tryton
Enter password for new role:
Enter it again:
$ exit
dale@casper:~ %

The “tryton” super-user password will ned to be entered in trytond.conf (the trytond daemon configuration file) for Tryton to access its database.

Restart PostgreSQL.

dale@casper:~ % sudo /usr/local/etc/rc.d/postgresql restart
LOG:  ending log output to stderr
HINT:  Future log output will go to log destination "syslog".
dale@casper:~ %

Confirm localhost can connect to the PostgreSQL server (“-W” causes psql to prompt for the user password).

dale@casper:~ % psql --username=tryton -W --list
Password for user tryton:
List of databases
Name       | Owner    | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+---------+-------+-----------------------
postgres   | postgres | UTF8     | C       | C     |
template0  | postgres | UTF8     | C       | C     | =c/postgres +
           |          |          |         |       | postgres=CTc/postgres
template1  | postgres | UTF8     | C       | C     | =c/postgres +
           |          |          |         |       | postgres=CTc/postgres
(3 rows)
dale@casper:~ %

Install Tryton Dependencies

Install Python and other dependencies using pkg.

% sudo pkg install python
% sudo pkg install py36-pip
% sudo pkg install bash
% sudo pkg install py36-virtualenv
% sudo pkg install py36-virtualenvwrapper
% sudo pkg install libxml2
% sudo pkg install libxslt
% sudo pkg install graphviz

Install the Python pydot package, the Python interface to Graphviz, and psycopg2, the Python interface to PostgreSQL.

% sudo pip install pydot
% sudo pip install psycopg2

Create a trytond system user

The “tryton” system user will execute the trytond daemen. The tryton user’s home directory (/home/tryton) will be configured as the root of the Tryton file system for storing document attachments. 

dale@casper:~ % sudo adduser
Username: tryton
Full name: trytond system user
Uid (Leave empty for default):
Login group [tryton]:
Login group is tryton. Invite tryton into other groups? []:
Login class [default]:
Shell (sh csh tcsh bash rbash nologin) [sh]:
Home directory [/home/tryton]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]: no
Lock out the account after creation? [no]: no
Username   : tryton
Password   : <disabled>
Full Name  : tryton system user
Uid        : 1002
Class      :
Groups     : tryton
Home       : /home/tryton
Home Mode  :
Shell      : /bin/sh
Locked     : no
OK? (yes/no): yes
adduser: INFO: Successfully added (tryton) to the user database.
Add another user? (yes/no): no
Goodbye!
dale@casper:~ % 

Create a Tryton source directory

Create a sub-child directory in  the tryton user directory for Tryton source code.

dale@casper:~ % sudo mkdir /home/tryton/tryton
dale@casper:~ %

Configure a virtual environment

Create a “default” virtual environment to use for execution.

dale@casper:~ % su - Password:
root@casper:~ # bash
[root@casper /root]# cd /home/tryton/tryton 
[root@casper /home/tryton/tryton]# virtualenv default
Using base prefix '/usr/local'
New python executable in /usr/home/tryton/default/bin/python3.6
Also creating executable in /usr/home/tryton/default/bin/python
Installing setuptools, pip, wheel...done.
[root@casper /home/tryton/tryton]# source default/bin/activate
(default) [root@casper /home/tryton/tryton]# 

Install trytond and modules

Install trytond – the Trytond server daemon.

(default) [root@casper /home/tryton/tryton]# pip install trytond

trytond modules provide areas of functionality to trytond. You can generate a list of all available modules using pip.

(default) [root@casper /home/tryton/tryton]# pip search tryton

Install the trytond_sale module to provide a basic set of functionality.

(default) [root@casper /home/tryton/tryton]# pip install trytond_company

Modules often have dependencies. To see all the trytond modules which were installed, you can use pip to list installed modules and grep to filter for trydond.

(default) [root@casper /home/tryton/tryton]# pip list | grep trytond
trytond 5.2.6
trytond-account 5.2.3
trytond-account-invoice 5.2.2
trytond-account-invoice-stock 5.2.0
trytond-account-product 5.2.0
trytond-company 5.2.0
trytond-country 5.2.0
trytond-currency 5.2.1
trytond-party 5.2.0
trytond-product 5.2.1
trytond-sale 5.2.0
trytond-stock 5.2.1
(default) [root@casper /home/tryton/tryton]#

Create log directory for trytond

dale@casper:~ % sudo mkdir /var/log/trytond
dale@casper:~ % sudo chown -R tryton:tryton /var/log/trytond

Create json-rpc data directory for trytond

dale@casper:~ % sudo mkdir /var/run/trytond
dale@casper:~ % sudo chown -R tryton:tryton /var/log/trytond

Create trytond.conf

The Tryton configuration file trytond.conf is read by the Tryton server daemon trytond when it starts, and includes such site-specific data as:

  • computer addresses to respond to (jsonrpc).
  • username and password for the PostgreSQL “tryton” super-user.
  • Tryton “administrator” password (required to create, drop, backup or restore a database).
  • specify FreeBSD-specific directory paths

Unfortunately, the pip install for trytond currently does not copy the default trytond.conf file in the package to a suitable location (or rather, to any location). Download the trytond server distribution from PyPi, extract and copy trytond/etc/trytond.conf to /usr/local/etc/trytond.conf.

Edit parameters in /usr/local/etc/trytond.conf using the following as reference:

jsonrpc = *:8000,0.0.0.0:8000
jsondata_path = /var/run/trytond

db_type = postgresql
db_host = localhost
db_port = 5432
db_user = tryton
db_password = appleton

admin_passwd = appleton

pidfile = /var/run/trytond/trytond.pid
logfile = /var/log/trytond/trytond.log

data_path = /home/tryton

Create tryton rc script

An rc.d script will be created to manage starting and stopping trytond, and to start trytond after booting. I’m using a basic rc.d startup script created by Christoph Larsen for the GNU Health project, and added support for status reporting. Copy the following code to /usr/local/etc/rc.d/trytond

#!/bin/sh

#
# PROVIDE: trytond
# REQUIRE: DAEMON
# BEFORE:  LOGIN
#
# Originally created by: Christoph H. Larsen
# http://lists.gnu.org/archive/html/health-dev/2011-11/msg00008.html
#

. /etc/rc.subr

name=trytond
rcvar=`set_rcvar`

load_rc_config $name

: ${trytond_enable="NO"}
: ${trytond_user="tryton"}
: ${trytond_group="tryton"}

start_cmd=${name}_start
stop_cmd=${name}_stop
restart_cmd=${name}_restart
status_cmd=${name}_status

command="/usr/local/bin/trytond"
required_files="/usr/local/etc/trytond.conf"

trytond_start() {
  su tryton -c "$command --config=/usr/local/etc/trytond.conf" &
}

trytond_stop() {
if [ -f /var/run/${name}/${name}.pid ]; then
  kill `cat /var/run/${name}/${name}.pid`
  fi
}

trytond_restart() {
  if [ -f /var/run/${name}/${name}.pid ]; then
  kill `cat /var/run/${name}/${name}.pid`
  sleep 1
  fi
  su tryton -c "$command --config=/usr/local/etc/trytond.conf" &
}

run_rc_command "$1"

Make /usr/local/etc/rc.d/trytond executable

# chmod u+x /usr/local/etc/rc.d/trytond

Add trytond_enable to /etc/rc.conf

# echo "trytond_enable=YES" >> /etc/rc.conf

Start trytond

Start trytond using the rc script:

# /usr/local/etc/rc.d/trytond start

Once started, you can check if trytond is running with the status option:

# /usr/local/etc/rc.d/trytond status

or stop the server if needed:

# /usr/local/etc/rc.d/trytond stop

Create and configure a new Tryton database

Download and install the Tryton desktop client for your system. Launch the Tryton client and access menu: File > Databases > New database.

Enter the Tryton server admin password (“admin_passwd” in trytond.conf) in the password field, then click Change beside the IP address and change the address to that of your server.

You must enter the password first, before changing the server address, because the Tryton client will attempt to connect to the server immediately after the server address is changed, and will report “Unable to connect” if the admin password was not already entered.

Enter the name of the database to create (e.g. “scc”) and the admin password for the database, then click Create.

Login to the new database as user “admin” and the password you entered to create the database. The Module Configuration Wizard will run automatically after login to configure the new database.

  • Add user (e.g. “Dale Scott”, login “dale”), optionally add permissions (you may need to update user permissions after installing modules, so this is optional at this point),
    • add Permissions: “Administration” (which will be the only permission group available)
    • add Rule: Read, Write, Create, Delete, Model: View Search (which will be the only rule available available)

Install Tryton modules into the database

Mark modules for install

The Tryton modules previously installed to the server will not be installed in a Tryton database.

In the Tryton client, access the menu: Administration > Modules > Modules. Mark the following modules for install (double-click in the Mark for Install column):

  • dashboard
  • product
  • product-attribute
  • production
  • project
  • project-plan
  • purchase
  • sale
  • stock
  • stock-lot
  • stock-split

Install marked modules

There are two ways to install the marked modules, you can either:

  • Select Launch Action icon in the Modules menu bar and then Perform Pending Installation/Upgrade, or
  • Execute the Perform Pending Installation/Upgrade Wizard from the main Tryton menu.

After the install/upgrade completes, the Module Configuration wizard runs and will query you to configure the installed modules.

Create a Company

A company is a type of ‘party’. To identify a party as a company in a new database you must first create a new party, then select it as the new company.

  • Name
    • Swift Construction Company
  • Addresses
    • Name: Default
    • Street: 1 Swift Way
    • Zip: 13054
    • City: Shopton
    • Country: United States
    • Subdivision: New York
  • Language
    • English

Configure Company

  • Enter Company > Currency
    • US Dollar
  • On the Company > Employees > Party view
    • select the new company (e.g. Swift Construction Company)
    • select Add

Create Chart of Accounts

  • Company: Swift Construction Company
  • Account Template: Minimal Account Chart
  • Create Default Properties
    • Default Receivable Account: Main Receivable
    • Default Payable Account: Main Payable

You will need to reload the menu in the Tryton client if it isn’t now displaying menu choices for the new modules (i.e. access menu: User > Menu Reload (Ctrl-T).

The Tryton server has been installed and you can connect to it from a Tryton client. You have also created a new Tryton database with the modules necessary to implement Maestro workflows.

Miscellaneous

Update FreeBSD periodic databases

I don’t know if installing packages triggers the periodic databases to update, but since I often don’t run a virtual machine long enough to cross day, week or month boundaries (when they update automatically), I typically manually update them after major system changes.

# periodic daily
# periodic weekly
# periodic monthly

References