Vital AI Tech Toolkit

Mastering software engineering empowers practical Artificial Intelligence (AI) utilisation, as it fundamentally relies on software code; selecting the suitable AI technology stack is vital for business and productivity automation.

Understanding AI’s underlying algorithms and data structures is essential for creating robust and efficient solutions. Developers and SMEs can leverage this knowledge to develop bespoke AI applications that can significantly boost operational efficiency and drive innovation.

A typical business stack for a small services-based business would look like this:

# Mini-Chat-CRM System for Dental Surgeons powered by AI.

# Functions: 

- Inform and educate patients.
- Facilitate patient onboarding.
- Schedule appointments.
- Enable chat for queries and support.
- Send updates and notifications.
- Process payments.

# Communication: Supports multiple channels (at least two).

Software engineering comprises three components: software environment, business stack, and code. Similarily, a good developer would analyse at least three contrasting technology stacks to introduce AI-driven automation.

A typical technology stack for a small services-based business would look like this:

# AI Technology Stack Overview:

# Environment:
# Development: Linux, Unix, Windows, MacOS, Docker.
# Production: Cloud platforms include Azure, AWS, Google Cloud, 
# Digital Ocean, plus any light dedicated servers.

# Programming Languages:
# - Includes C/C++, Python, Java, .NET, JavaScript, Ruby, and PHP.
# - Ensures versatility across different software projects.

# Version Control:
# - Utilises Git, GitHub, GitLab, Bitbucket, and Mercurial.
# - Facilitates effective source code management.

# Databases:
# - Incorporates PostgreSQL, MySQL, MongoDB, Redis, and Cassandra.
# - Offers robust data storage and management solutions.

# User Interfaces:
# - Dynamic interaction through HTML, CSS, and 
# - JavaScript or static site engines like Hugo.
# - Enhances frontend and backend user experiences.

# Payment Gateways:
# - Integrates Stripe, PayPal, WorldPay, and Adyen for transactions.
# - Provides secure and reliable financial processing.

# Communication Channels:
# - Expands to the Web, mobile apps, and Twilio for diverse outreach.
# - Ensures voice, text, WhatsApp capabilities and alternatives.

# Analytics and Conversation:
# - Employs Google Analytics and chatbots for insights and interaction.
# - Enhances user engagement and data analysis.

# QA and Testing:
# - Adopts Selenium, JUnit, and TestNG for comprehensive testing.
# - Ensures software reliability and performance.

# Additional Libraries:
# - It enriches with NLP, LLM, ML, data analysis, and visualisation.
# - Advances AI capabilities with intent and sentiment detection.

# Security and Compliance:
# - Implements OAuth for authentication and GDPR for data protection.
# - Prioritises user privacy and cyber security standards.

# Cloud Services and Infrastructure Management:
# - Droplets and Kubernetes are for container orchestration, 
# - Terraform is for IaC,
# - Streamlines application management and scalability.

To harness AI, SMEs must adopt a streamlined technology stack. Utilising minimal tools maximises AI’s potential, improving efficiency and utility. Refer to How SMEs Can Leverage AI for further insights.

Discover the latest repos in the industry

Explore top-tier AI technology stacks and GitHub repositories, focusing on open-source projects in nlp, chat, backend systems, frontend funnels, visualisation, machine learning, data science, and Mini-CRM systems.

These resources are specifically designed to boost the efficiency of developers and researchers, facilitating AI-driven automation for SMEs. By leveraging these advanced technology stacks, SMEs can unlock powerful automation capabilities, streamline operations, and enhance customer engagement. This curated selection supports the development and research communities deploying innovative AI solutions.

Software Environment Setup

A well-configured software environment ensures applications run consistently, embodying best practices by minimising compatibility issues. Containerisation and virtualisations are pivotal for robust environments, offering efficient application management and resource utilisation. These technologies streamline deployment, enhance team collaboration, and accelerate software delivery.

Software Environment Setup

# A. Containerisation

- Overview: Packages software and its operating environment into a container
- Use Case: Ideal for deploying and managing microservices,
  enhancing portability and efficiency

- Tools: 
  - Docker
  - Kubernetes
  - OpenShift
  - Docker Swarm
  - Rancher

Docker Github: 
https://github.com/docker/compose
https://github.com/docker/getting-started

For virtualisation tools

# B. Virtualisation

- Overview: To run multiple operating systems on one physical server
- Use Case: Suitable for development environments or server consolidation

- VMware vSphere: Premier platform for virtualised infrastructure 
https://www.vmware.com/products/vsphere.html

- Oracle VirtualBox: Open-source for diverse OS virtualisation
https://www.virtualbox.org/

- Hyper-V: Windows-based solution for virtual machines
https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/about/

- KVM: Linux kernel virtual machine technology 
https://www.linux-kvm.org/

- Xen: Secure, efficient open-source hypervisor
https://xenproject.org/

Programming Languages

1. Python

Python is a powerful tool for developers due to its simplicity, readability, elegant syntax, dynamic typing, and a vast collection of libraries and frameworks. Python’s ecosystem is versatile and capable of adapting across diverse computing disciplines, from web applications with Django and Flask to complex data analysis with NumPy, Pandas and pioneering AI innovations through TensorFlow and PyTorch.

Python’s active community enriches its ecosystem, making it adaptable for beginners and experienced developers.

Python: Comprehensive libraries for AI and data analysis
https://www.python.org/about/gettingstarted/

### Python Run & Compile
- Env: Python Interpreter
- Compile: Interpreted, JIT via PyPy
- JIT instantly translates code to run faster

# Python Installation
sudo apt-get update
sudo apt-get install python3

# Django: Web development, full-stack
# Install Django:
pip install django

# Flask: Web development, lightweight
# Install Flask:
pip install flask

# ML & Data Science Libraries

# NumPy: Data science, numerical computations
# Install NumPy:
pip install numpy

# Pandas: Data science, data manipulation
# Install Pandas:
pip install pandas

# TensorFlow: Machine learning, deep learning
# Install Tensorflow:
pip install tensorflow

# PyTorch: Machine learning, neural networks
# Install PyTorch:
pip install torch torchvision torchaudio
Python is central to driving innovations and solutions in data science, machine learning, web development, and automation. To get started with Python and its vast ecosystem, here are some basic terminal commands for installation:

2. R

R is a potent tool for statistical computing, data analysis, and visualisation. Its comprehensive packages make it an essential resource for data scientists and statisticians.

The R ecosystem, supported by CRAN, includes popular libraries like ggplot2, dplyr, shiny, and caret for data visualisation, manipulation, web apps, and machine learning.

R’s community continually improves capabilities for research and industry.

R: Statistical analysis and graphics.

R: Statistical computing and graphics
https://www.r-project.org/other-docs.html

### R Run & Compile
  - Env: R Interpreter
  - Compile: Interpreted, some JIT options

# Install R
sudo apt-get update
sudo apt-get install r-base

# R Packages
# ggplot2: Data visualisation
install.packages("ggplot2")

# dplyr: Data manipulation
install.packages("dplyr")

# shiny: Interactive web apps
install.packages("shiny")

# caret: Machine learning
install.packages("caret")

3. Java

Java is a versatile programming language for building enterprise-scale apps, web servers, mobile apps, and desktop applications. Java is an object-oriented language around objects allowing developers to structure code more organised and modularly.

Java is highly portable and provides developers with powerful tools like Spring, Apache Struts and Hibernate.

Java’s platform independence through the JVM (Java Virtual Machine) allows its applications to run on any device or operating system, making it highly portable and widely adopted.

Java is a compiled language, while JavaScript is interpreted.

Java is Known for its robustness 
and portability in application development.
https://docs.oracle.com/en/java/

### Java Run & Compile
- Env: Java Virtual Machine (JVM)
- Compile: Compiled to bytecode, runs on JVM

# Installation Instructions for Java 
To install Java on your system, follow these steps:

1. Update your package index:
sudo apt-get update

2. Install the default Java Development Kit (JDK):
sudo apt-get install default-jdk

Once installed, verify the installation by:
java -version

// Sample Java code for a simple "Hello, World!" program
public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello, World!");
    }
}

4. JavaScript

Integral for creating dynamic and interactive web pages, JavaScript thrives with broad browser compatibility and a vast ecosystem of libraries. However, its loosely typed nature can lead to complex debugging processes.

- JavaScript
  https://developer.mozilla.org/en-US/docs/Web/JavaScript

### JavaScript Run & Compile
- Env: Browser's JS engines (V8, SpiderMonkey)
- Compile: Interpreted in browsers/Node.js

5. C#

C Sharp favoured for its robustness in developing a wide range of applications and seamless integration with the .NET framework, C# ensures type safety and scalability. Its platform dependency on Windows, however, could be restrictive.

- C#
  https://docs.microsoft.com/en-us/dotnet/csharp/

### C# Run & Compile
- Env: .NET Runtime / .NET Core Runtime
- Compile: Compiled to Intermediate Language (IL)
- JIT compiled to native code at runtime

6. C++

Celebrated for its unparalleled performance and control over system resources, C++ is ideal for developing game engines and software requiring high-efficiency execution. The complexity of its syntax, though, demands a steep learning curve.

- C++
  https://isocpp.org/

### C++ Run & Compile
- Env: System OS
- Compile: Compiled to machine code

7. TypeScript

Enhancing JavaScript with static typing, TypeScript offers improved code quality and predictability, particularly in large-scale applications. Nonetheless, the initial setup introduces an additional layer of complexity.

- TypeScript
  https://www.typescriptlang.org/

### TypeScript Run & Compile
- Env: Depends on target platform (browsers, Node.js)
- Compile: Transpiled to JavaScript

8. Ruby

Ruby’s elegant syntax and the robust Rails framework facilitate rapid web development and prototyping. Yet, runtime performance can lag, especially in comparison to compiled languages.

- Ruby
  https://www.ruby-lang.org/en/

### Ruby Run & Compile
- Env: Ruby Interpreter (MRI, JRuby, etc.)
- Compile: Interpreted, with some 
- JIT options in recent versions

9. C

As the cornerstone of modern programming, C offers direct manipulation of hardware and efficient execution of applications. However, its lack of built-in safety features poses a risk of errors and vulnerabilities.

- C
  https://en.wikipedia.org/wiki/C_(programming_language)
  Note: C doesn't have official website

### C Run & Compile
- Env: System OS
- Compile: Compiled to machine code

10. Swift

Swift’s modern syntax and focus on safety make iOS and macOS development more intuitive and less prone to errors. Limited use outside Apple’s ecosystem could be a drawback.

- Swift 
  https://swift.org/

### Swift Run & Compile
- Env: Apple platforms, Linux
- Compile: Compiled to optimized native code

11. Objective-C

Once the primary language for Apple applications, Objective-C provides direct access to C libraries and runtime flexibility. Its syntax and legacy status, compared to Swift, may deter new developers.

- Objective-C 
[Documentation] (https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/Introduction/Introduction.html)

### Objective-C Run & Compile
- Env: Apple platforms
- Compile: Compiled to machine code, with runtime for dynamic typing

12. Scala

Scala offers a blend of object-oriented and functional programming paradigms, ensuring concise code and immutability. The complexity of its features, however, can intimidate beginners.

- Scala
  https://www.scala-lang.org/

### Scala Run & Compile
- Env: Java Virtual Machine (JVM)
- Compile: Compiled to Java bytecode

13. Go

Go known for its simplicity and efficient concurrency handling, Go (or Golang) is optimised for modern computing architectures. Its simplicity, paradoxically, limits expressiveness in complex applications.

- Go 
  https://golang.org/

### Go Run & Compile
- Env: System OS
- Compile: Compiled to machine code

14. Kotlin

Kotlin’s safety features and interoperability with Java make it a favoured choice for Android development, reducing boilerplate code. Compared to Java, the smaller community might affect learning and troubleshooting resources.

- Kotlin
  https://kotlinlang.org/

### Kotlin Run & Compile
- Env: JVM, Android, Native platforms
- Compile: Compiled to Java bytecode or native binaries

15. Rust

Rust guarantees memory safety without a garbage collector, ideal for system programming and high-performance applications. Its steep learning curve and meticulous ownership model can be challenging for newcomers.

- Rust
  https://www.rust-lang.org/

### Rust Run & Compile
- Env: System OS
- Compile: Compiled to machine code

16. Dart

Dart excels in developing visually attractive and natively compiled multi-platform applications, especially with the Flutter framework. Dart’s niche adoption outside Flutter development is a notable limitation.

- Dart
  https://dart.dev/

### Dart Run & Compile
- Env: Dart VM, browsers (via compilation to JavaScript), Flutter
- Compile: JIT in development, AOT for production

17. Perl

Perl’s text processing capabilities and comprehensive CPAN repository make it powerful for scripting and backend development. However, its syntax, described as cryptic, complicates readability and maintenance.

- Perl 
  https://www.perl.org/

### Perl Run & Compile
- Env: Perl Interpreter
- Compile: Interpreted

18. Lua

Lua’s lightweight footprint and embeddability perfectly suit game development and embedded systems. Its limited standard library, requiring external modules for everyday tasks, is a drawback.

- Lua
  https://www.lua.org/

### Lua Run & Compile
- Env: Lua Interpreter, embedded in applications
- Compile: Interpreted, JIT via LuaJIT

19. Haskell

Haskell’s pure functional programming model encourages high code correctness and abstraction. However, the significant paradigm shift from imperative programming presents a considerable learning obstacle.

- Haskell
  https://www.haskell.org/

### Haskell Run & Compile
- Env: System OS via Glasgow Haskell Compiler
- Compile: Compiled to machine code

20. .NET

.NET is a free, open-source platform by Microsoft for building various applications. It works across the web, mobile, desktop, games, and IoT. You can use multiple languages, editors, and libraries. The framework is popular due to its comprehensive ecosystem, robust security, and regular updates.

- .NET 
  https://dotnet.microsoft.com/

### .NET Run & Compile
- Env: .NET Runtime / .NET Core Runtime
- Compile: Compiled to Intermediate Language (IL), 
- JIT compiled to native code at runtime

It is a preferred choice for enterprise-level projects due to its scalability and high-performance capabilities. However, developers heavily invested in other technology stacks may face limitations due to its close association with the Microsoft ecosystem.

21. Flutter

Flutter is an open-source UI development kit by Google used to create apps for multiple platforms from a single codebase.

- Flutter
  https://flutter.dev/

### Flutter Run & Compile
- Env: Dart VM for development; iOS and Android for production
- Compile: AOT compiled to native code for production

Flutter is fast, flexible, and offers native performance. It creates visually attractive, natively compiled applications quickly. The learning curve for Dart and the platform’s young age may be minor drawbacks.

22. COBOL

COBOL, the cornerstone of business computing, is integral to financial systems globally. Its strength lies in processing extensive transactions reliably, not in processing speed, marking its significance in the banking industry where precision is paramount.

- COBOL
  https://www.microfocus.com/en-us/products/cobol-development-and-mainframe-solutions/overview.

### COBOL Run & Compile
- Env: Mainframe systems, COBOL runtime environments
- Compile: Compiled to machine code

Cobol’s resilience is supported by platforms like Micro Focus, ensuring COBOL’s compatibility with modern technologies.

23. FORTRAN

FORTRAN remains a stalwart in scientific computing, revered for its precision in numerical analysis and computation. It’s a key player in fields demanding mathematical calculations, supporting groundbreaking research and explorations.

- FORTRAN
  https://fortran-lang.org

### FORTRAN Run & Compile
- Env: System OS
- Compile: Compiled to machine code

Despite the emergence of newer programming languages, FORTRAN’s specialised libraries, like BLAS and LAPACK, underscore its ongoing importance in scientific domains. Fortran Lang offers comprehensive resources for the latest on FORTRAN’s evolution and its community.

24. Pascal

Pascal shines in educational contexts, promoting strong programming fundamentals that have informed the design of contemporary languages. Its legacy endures through the Free Pascal Compiler, bridging Pascal’s principles with modern cross-platform development.

- Pascal
  https://www.freepascal.org.

### Pascal Run & Compile
- Env: Depends on compiler 
- (e.g., Free Pascal Compiler)
- Compile: Compiled to machine code or bytecode

Though its professional application has waned, Pascal’s emphasis on clarity and structure continues to influence new generations of programmers and software engineers.

25. PHP

PHP powers billions of websites, including WordPress, Drupal and Joomla. It’s easy to use, flexible, and has powerful frameworks like Laravel and Symfony.

PHP frameworks have transformed web development with elegant syntax, robust features, and comprehensive tools for routing, authentication, and caching tasks.


- PHP
  https://php.net/

### PHP Run & Compile
- Env: Web servers with PHP installed (e.g., Apache, Nginx)
- Compile: Interpreted

- Laravel
  https://laravel.com/

- Symfony
  https://symfony.com/

- Zend (Now Laminas Project)
  https://getlaminas.org/
  

Laravel and Symfony are popular PHP frameworks that contribute to PHP’s role in the modern web. Both frameworks make it easier to integrate AI into PHP projects. Tools are available to leverage PHP’s scripting capabilities for data-driven features.

PHP is an evolving programming language that adapts to new challenges. Recent updates focus on improving performance, security, and modern language features. The PHP community fosters a robust ecosystem for developers, making it a key player in web development.

Programming Languages Summary

Python, PHP, Java, R, and Ruby stand out in software development, each tailored to specific tasks: Python excels in AI and data science; PHP dominates web development; Java is universal, especially for enterprise; R leads in statistical analysis; Ruby offers simplicity in web projects. Their unique strengths guide developers in choosing the right tool for their project’s goals and team skills, ensuring a blend of innovation and practicality.

Swift emerges as a critical player in Apple app development, offering a modern, fast, and safe programming environment. Swift’s compatibility with Apple’s ecosystem allows developers to leverage native libraries and tools for creating intuitive and high-performance iOS applications.

Kotlin, recognised for its safety and conciseness, is the counterpart for Android app development. It provides a seamless development experience, reducing the boilerplate code and enhancing app performance and maintainability on Android devices.

On the web development front, PHP and .NET provide robust backend solutions, facilitating server-side scripting and application logic for web applications. PHP’s wide use in web development is complemented by. NET’s framework for building efficient, secure web apps and services.

Flutter introduces a modern framework for developing natively compiled mobile, web, and desktop applications from a single codebase. Its ability to deliver high-quality, multi-platform applications with a single development process makes it an invaluable tool for seamlessly extending software across various platforms.

Databases

In software development, databases are crucial architectural elements influencing application functionality and user experience. They facilitate electronic storage, manipulation, and information retrieval, making them indispensable for various systems. Database Management Systems (DBMS) ensure data remains organised and accessible, bolstering operational efficiency and scalability across diverse sectors.

Choosing a DBMS is critical and impacts an application’s architecture and future development. Here are premier DBMS options with unique capabilities and use cases in software development.

1. PostgreSQL

PostgreSQLis a highly versatile and compliant SQL database. It excels at managing complex data workloads and large datasets, making it ideal for enterprise-grade applications.

sudo apt-get update
sudo apt-get install postgresql postgresql-contrib

2. MongoDB

MongoDB is known for its schema-less data model that enables flexibility and rapid development, especially in big data and applications requiring scalable document storage.

wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
echo "deb http://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list
sudo apt-get update
sudo apt-get install -y mongodb-org

3. MySQL

MySQL is a reliable and efficient option for managing simple data storage but has limitations under high-load conditions.

sudo apt-get update
sudo apt-get install mysql-server

4. Redis

Redis is great for rapid data access like caching and real-time analytics, but not ideal for persistent data management due to its reliance on memory storage.

sudo apt-get update
sudo apt-get install redis-server

5. SQLite

SQLite is a lightweight embedded database ideal for mobile and small web projects, but lacks concurrent processing capabilities. Evaluate DBMS based on data model complexity, scalability demands, and specific functional requirements of the application.

sudo apt-get update
sudo apt-get install sqlite3

6. Neo4j

Neo4j provides powerful graph database functionalities, while Couchbase offers versatile NoSQL options for web and mobile apps.

Neo4j for Linux: official package. 
More options on their website.
https://neo4j.com/docs/operations-manual/current/installation/

7. Cassandra

Cassandra is good for scalability in distributed settings. MariaDB is recommended for advanced features. Oracle and SQL Server are suitable for enterprise data management.

Cassandra can be installed on Linux systems via the command line. 
https://cassandra.apache.org/_/download.html

8. Amazon’s DynamoDB

Amazon’s DynamoDB is a fully managed NoSQL service designed for high performance and scalability in cloud environments.

DynamoDB is accessible as a cloud service. 
https://aws.amazon.com/dynamodb/.

Development & Collaboration Tools

Please explore our collection on getting started with software engineering, covering essential development and collaboration tools.