Using htop to explore server performance

I recently replaced the server for dalescott.net after a drive failure using a faster system, but my initial perception was that it was slower!

The “new” server is a Intel Core2 CPU 6600 2.40GHz with 6GB of RAM. In addition to the services running on bare metal,  VirtualBox is used to host ERPNext using one core and 2GB RAM. It was the ERPNext response that seemed particularly slow. Investigating, I found a combination reasons as is typical of most problems.

First, I examined the apache logs and found the server was the target of a penetration attack around the time I had noticed response was “slow”. The server regularly shows 1000+ penetration attempts in a 24hr period, which tend to occur in clusters. However they have been more an annoyance than a significant concern, as I had not noticed them having a significant effect on server performance.  Next, I examined virtual machine performance and found CPU use was 100% when ERPNext was being accessed. 

While not finding a definitive cause, it seems likely the additional CPU load from the penetration attack was starving the already high CPU requirements of ERPNext.  While clearly ERPNext is processor-constrained, under normal load the server can still provide adequate performance for basic presentations or training with a limited number of concurrent users. However, not if someone is knocking on the front door trying to break in at the same time. 

dalescott.net SaaS Architecture

Load At Rest

Below is the virtual machine and host server when at rest (no logged-in ERPNext users, and no on-going brute-force ssh or web app login attempts). The vm load (top) is 3% and the host (bottom) server cores are 5% and 2% (the 2% core is the erpnext server).

Load When Stressed

Below is the server when an ERPNext user logs in and accesses an Item list. The vm load (top) has maxed out at 100% and is using half its available memory. On the host (bottom), core #2 is almost 70% (erpnext), but core #1 is only 35% and only half the total available memory is being used, which is good.

 

Credential Stuffing

dalescott.net is a regular target for credential stuffing penetration attacks. I took a closer look at last nights attempted login IDs. By it’s appearance, “1234” and “m” must be popular with sysadmins for quick login ID (but not for me ;-)).

1234
aaron
account
adam
adm
admin
administrateur
administrator
apache
asteriskftp
auction
backup
boss
centos
cisco
connect
console
contact
cop
demo
D-Link
ems
fax
ftp
ftpuser
guest
helpdesk
info
installer
jenkins
kelly
kobayashi
lock
logout
lpa
lpd
luanvandiemcao
luckup
lucky
m
mailman
mailtest
manager
marketing
monitor
msf_user
nagios
network
new
office
openfiler
oracle
password
pi
PlcmSpIp
post
postfix
postgres
remote
report
sales
scan
scanner
sconsole
software
spam
squid
sue
support
svn
sync
synopass
system
tamaichi
test
test01
testuser
tomcat
ubnt
ubuntu
uftp
unknown
unlock
uploader
url
user
uucp
vagrant
vmail
vpn
webmaster
wordpress
www-data

ERP, IIoT and MQTT

MQTT represents a paradigm shift in industrial process control communication, is the primary communication protocol for IoT, and is anticipated to be the standard communication protocol for networked industrial process control devices – the IIoT.

A strategic opportunity may exist for an early market technology leader with a proven low-cost operations stack consisting of IIoT MQTT endpoints (with optional “intelligence at the edge”) providing data which directly result in the creation of orders or material movements in an ERP system, at a cost acceptable to SMEs. MQTT is still new technology for process control, and significant interest may also come from large enterprise investigative projects as they learn and find new strategies.

End-user MQTT References:

Video and podcast interviews with Arlen Nipper, co-creator of MQTT.

 

What is Open-Source?

I like to think of Open Source as a legal framework that supports shared software development by a community. The license protects the investment made by the community (time coding, testing, writing bug reports, writing user manuals, supporting other members of the community, etc.), as well as protecting the community from an actor acting in bad faith.

When a company (a legal entity) develops software, the code generally belongs to the company, regardless of whether the programmer is a full-time employee or a contractor, and the developer is not entitled to a single character. Of course, developers still own the knowledge in their heads, and they can use that knowledge to re-create equivalent functionality (although other contractual restrictions, such as non-competition and non-disclosure agreements, may still apply).

A single developer providing software under an open source license is generally simply being altruistic and helping their fellow developers by providing tutorials, examples and proof-of-concepts consistent with a single developer project. The developer’s personal beliefs will direct the license, either “permissive” (e.g. the BSD license) or “copyleft” (e.g. the GPL).

A permissive license might be appropriate if the goal is to provide benefit to as many people as possible with minimum constraints (a permissive license typically only imposes keeping the original license and copyright notice, and prevents being sued for errors or not being fit for use).

However, if you believe someone who modifies your code has an obligation to share in kind, then imposing this through a copyleft license will likely be appropriate. Note though that the GPL only requires source to be shared with those who receive the software in non-source form, which may not include the original developer! Payment is irrelevant as far as the license is concerned.

The license becomes more significant for projects with multiple developers, projects that incorporate open-source software to expedite development, and companies who use the software. Vague ownership of the code or vague statements of allowed use creates risks for the entire community.

What if two developers work together for a year to create a software application but then part ways. Who owns the rights to the codebase? Do the developers share ownership jointly or individually? Can one developer continue development of the software on their own if the other developer doesn’t want them to? Does each developer have rights only to the characters they typed?

Vague ownership is also a risk for users if the project doesn’t have the legal right to offer the software, which could caused by something as simple as including a GPL library in a BSD project, using code available publically without authorization from the author, or a developer re-using old project code that was written for an employer instead of recreating it from scratch. If the software is an enterprise ERP, CRM or FRACAS system, extracting it from the business could be difficult and costly. Any company that proactively protects shareholders from risky legal situations would have to just walk away before even starting.

Licensing itself isn’t complicated, it’s all the other details…. 😉