January 2019

Advantages of implementing Big Data in your company

Big Data is not a ‘trend’. It is a necessity associated with most large, or even medium or small, companies that can no longer get sufficient value from the data produced by more traditional Business Intelligence tools. Big Data plays an important role in boosting business, and many companies are already aware of that. According to Forbes, the global market for Big Data (software and services) will grow from 42 trillion dollars, in 2018, to 103 trillion dollars in 2027.

There are many advantages to implementing Big Data in your company, and having a well-defined strategy is a halfway house to being able to make well-informed decisions, which can be a key to success for your business.

What is Big Data?

Big Data is the ability to analyse and/or process very large amounts of data, based either on its volume or on the number of ‘data points’ generated. The concept of Big Data comes to the fore when companies face such a great flow of data that conventional processing and analysis tools cannot handle it effectively.

Data can be structured, semi-structured or unstructured. Structured data are, for example, data from purchases or sales from an organisation or information from forms or operational tables. Unstructured or semi-structured data are information generated without an established order and from sources such as, for example, social media, user logs in web or mobile apps, sharing of opinions or files.

According to data from Harvard Business Review, only 20% of the data that gets to companies is structured, while the other 80% is semi-structured or unstructured. Moreover, the percentage of that structured data that is used to support decision-making and to extract insights is less than 50%; however, for semi-structured or unstructured data, that percentage falls to 1%.

The Big Data concept can be characterised by five Vs:

  • Volume: massive amounts of data are generated and need to be stored and processed. According to the website Statista, already in 2018, 10.6 zettabytes were generated worldwide from cloud data centres.
  • Velocity: the velocity of generating, processing and analysing data can be more important than volume, since real-time or near-real time information provides great agility to companies that have a Big Data strategy implemented.
  • Variety: data can originate from various sources, such as normal data bases, social media, web pages, financial transactions, emails, sensors (IoT), audio, text or video files, archives, forums, etc.
  • Veracity: is the generated data reliable, according to its source or origin?
  • Value: do the generated data have true value for a company? It is necessary to assess if those data will, in fact, generate new opportunities, increase income or optimise costs, for example.

Advantages of implementing a well-defined strategy

So, we know that implementing a Big Data strategy has become a necessity for large organisations, and the focus has changed from “whether to use Big Data” to “how to use Big Data more efficiently”.

We also know that Big Data opens doors to better informed decision-making, based on extremely complex analysis, and that it allows the collection of important insights to optimise the information gathered. Consequently, the decision to implement a Big Data strategy must come from business teams and not from the IT departments that must ensure the technical execution of the project in the most efficient way. Basically, it is those business teams that will get value from the gathered data for their daily work and for the definition of  strategy.

However, what are the true advantages of implementing a Big Data project? What will be the advantages to the competitiveness of your business? We identify three of the main advantages of implementing Big Data in your company:

Advantage 1: Informed decision-making

With data analysis carried out by Big Data technologies, it is possible to find purchase or behaviour patterns that support decision-making from business departments. For example, if a marketing team has information that a certain family buys the same product every single month, it can send discounts for that same product through digital or physical mailing, in order to ensure that the customers stay faithful.

Advantage 2: Reduced costs

Data generated from or for a company are stored, processed and analysed, resulting in finding important business insights or the identification of gaps and errors. Working on data previously analysed and having access, for example, to constant behaviour or purchase trends, allows companies to launch more efficient campaigns that reach directly to the desired target and, therefore, can register a better ROI. This way, optimising the use of a budget will make teams more efficient – also increasing their productivity.

Advantage 3: Possibility to predict future situations

Usually, in Big Data, there are three types of analysis that can be carried out and complement each other:

  • Descriptive analytics, the type of analysis that describes what is happening, often in real-time. By the use of data aggregation and data mining, it is possible to access a picture from the past and understand the reason for a departure or a change – or just summarise a certain aspect.
  • Predictive analytics, the type of analysis that predicts what might happen in the future, relying on statistics and algorithms and providing scenarios of statistically probable situations.
  • Prescriptive analytics, based on optimisation, simulation algorithms, machine learning and computational models; this is quite a complex type of analysis, which seeks to answer the question “what should we do in a given situation?” Basically, the scenarios created will work as specifications of different actions and their expected outcomes, allowing the company to choose the scenario that represents least risk, for example.

Practical examples

Now that you know the advantages of implementing Big Data in your company and how to establish specific and measurable goals, the question is: how can you benefit from the data generated from the organisation, based on the area it impacts?

Here are a few practical examples:

  • Data from sensors in transportation systems;
  • Analysis of financial data to prevent fraud (for example, by detecting the use of a credit card from an unusual user);
  • Analysis of network traffic;
  • Monitoring mentions on social media to assess if the emotions towards a brand/company are positive or negative;
  • Information on traffic flows to predict which times will be more problematic.
Ana LamelasAdvantages of implementing Big Data in your company
read more

Tableau & Jira: A new way to look at your projects

Tableau is a self-service BI platform that allows the identification of valuable insights and provides advanced analysis, visualisations and the capability to share information quickly. As we all know, with the digital transformation era and with all the information surrounding our daily lives, the need to make decisions based on facts increases – this implies the ability to look at data and to be able to analyse and make decisions based on that data. Decisions taken on operational teams are no different, and in a service desk management team (and in bug fixing), having access to the numbers is the path to optimising teams and getting better results.

So, integrating Tableau and Jira is a new way to look at your projects because even though Jira offers some options to create reports or to get essential metrics, only with a tool such as Tableau will your team be able to cross-reference data with other data sources and create great looking dashboards or advanced analytics. This blog post aims to explain in detail how it is done.

Tableau is a visualisation tool, and it is divided into three modules: Tableau Desktop (allows the connection to all types of databases and enables the creation of business rules, field nomination and an overview of all data); Tableau Server (where you can publish views and share information with other team members – granting and removing access, writing comments and editing views); and Tableau Data Prep (which is an ETL tool that helps users prepare data and extract data from a variety of sources, transform that data and output it, saving much precious time).

Jira, as a project management tool, is not intended to analyse data in detail or to extract insights. It does have some features to create reports or to obtain some information: it has widgets; but if your team has different needs, for example, if your team truly needs to cross-reference data, you will need Tableau (because Jira can only access its own data). For example, to compare data from Jira with a timesheet application to see if the time registered in one app matches the time logged on the other app, you would need to install the All-In-One Tableau connector.

The All-In-One (AIO) Tableau connector is an app for Jira that implements a Web Data Connector (WDC) for Tableau. The WDC enables connections to data through HTTP when the data source does not have a Tableau native connector. The data is obtained and placed in an extract that becomes available to either Tableau Desktop or Server.

Setup

For an app to connect to Tableau through the WDC, you need to whitelist the corresponding URL available on the server. Make sure your sysadmin performs this step, and be aware that multiple keys will be generated, so the URL pattern and command need to be something along these lines:

tabadmin whitelist_webdataconnector -a  https://yourJIRAdomain/plugins/servlet/aio-tb/public/tableauconnect(.*)

Connecting to data

  1. Open Jira
    1. Obtain your AIO connector URL. Each URL provides access on behalf of a particular user, ensuring you will only have access to your own projects:
  1. On Tableau Desktop:
    1. Select the WDC connection type
    2. Paste the URL and click enter
    3. Define a name for this connection – perhaps the project name – and a JQL Query (to get all issues from a project, use: project = PROJECTID)
    4. Choose the fields or subjects of fields to retrieve, and click continue
    5. Depending on your selection, the WDC makes available multiple labels that you can now join as normal Tableau data sources
    6. Start your analysis
    7. Publish if you want to share

The result

You can now explore Jira data, create powerful dashboards, extract the most valuable insights and increase your team performance – all with a fantastic tool called Tableau!

Ana LamelasTableau & Jira: A new way to look at your projects
read more

Flutter – A broad Introduction

According to statista, by 2020, mobile apps are expected to generate around 189 billion US dollars in revenues via app stores and in-app advertising. Nowadays, companies see the mobile app not only as the business itself, but also as a means to advertise their brand, and are trying to reduce the time to market as much as possible in order to keep up with the competition.

Currently, there are two main operating systems running on smartphone devices: Android and iOS. Together, they make up almost 100% of the smartphones sold in the first quarter of 2018. These different platforms have their own design constraints, their own toolsets and their own programming languages, but each platform is merely building a user interface. Why should development across these two user surfaces be so different?

Many technologies have already addressed the necessity to unify development for the existing platforms, and names like Microsoft’s Xamarin and Facebook’s ReactNative have emerged as the most prominent cross-platform development solution.

Google is known for its awesome products but has never endorsed a long-term, supported, fully cross-platform project until recently. Read this Flutter introduction and discover all about the project.

1. What is Flutter?

According to Google, the enterprise behind the project, Flutter is a brand new technology that allows the crafting of high-quality native interfaces on iOS and Android with a single shared code base, using a programming language called Dart, also developed by Google.

With a free and open source SDK, Flutter is used by developers and organisations around the world and works with a centralised programming language and unified core. Additionally, Flutter integrates with standard, popular development environments and tools such as Visual Studio Code and IntelliJ.

This combination of key features enables fast development, a high performant result and high maintainability.

2. The Flutter project timeline

Flutter is quite new. Therefore, its timeline is fairly small and compact (for now!).

  • In early 2015, the project was up and running and known as ‘Sky’. It ran on Dart and was already capable of rendering screens at a 120 frames-per-second;
  • In 2016, Google unveiled their plans and codebase for a new operating system (OS) known as ‘Google Fuchsia’. This new OS is rumoured to be the successor to Android for universal devices, and its apps are entirely written in Flutter;
  • After maturing, the initial release took place in 2017. The alpha version, numbered as v0.0.6, was publicly available and developers could jump on this technology.
  • In May 2018, during Google IO, Flutter was finally shipped in its official beta release.
  • In September 2018, Google released the second preview of version 1.0.

Fuchsia is a new OS with a dedicated micro-kernel that runs on universal devices, from embedded systems to smartphones, tablets and personal computers. It is already clear that Flutter is not going to disappear or be abandoned any time soon. Google is boldly and consistently investing in Flutter, as can be seen from its presence in I/O 2017 and I/O 2018.

3. What makes Flutter different?

Flutter is not a new, ground-breaking, unseen solution. It is rather a new option for the competitive cross-platform world. But what value is Flutter bringing to this prolific environment? There are, of course, some points of differentiation.

Unlike ReactNative, which bridges Javascript to native code with a noticeable performance loss, or Xamarin.Android & Xamarin.iOS, which compile a single C# codebase into native code – thus requiring two completely distinct UI constructions – Flutter seems to overcome its competitors’ most common flaws. Skipping mid-level bridging and interpretation, leveraging powerful ahead-of-time compilation and making use of the flexible Skia Graphics Library, Flutter is all about widgets.

A widget represents an atomic area on the screen and its corresponding logic (e.g. Input Field, Button, Image, List). These small building blocks are fast, very responsive and customisable. Flutter’s solid and high-quality user interfaces come from the fact that you can combine, interweave and compose widgets in order to achieve larger, more complex screens.

Despite the existence of many community-developed widgets, Google’s Flutter team provides the most relevant ones. These widgets were built under Google’s material design or Apple’s Cupertino.

Another major advantage of Flutter is hot reload. This feature allows code to be incrementally added and executed in real time, instead of having to recompile code over and over again, reducing implementation and test time.

Just like most cross-platform technologies, Flutter allows interaction between its Dart codebase and native components, be they hardware or existing libraries. By leveraging this possibility, it can delegate difficult, expensive functionalities to native implementations and collect the results with great ease so they can be further used by the single codebase. For instance, it can use the battery service of each OS to obtain the device battery status.

3.1. Flutter vs. Xamarin

Xamarin.Android and Xamarin.iOS allow the developer to centralise code logic in a single C# codebase. This powerful feature prevents code duplication and minor logic bugs and speeds up the output process by saving development time. Nonetheless, in order to achieve platform dedicated UIs, the developer must implement the UI separately for each supported platform. Xamarin.Forms can help circumvent this necessity, but native UI implementation is usually the preferred option due to its flexibility.  Once ready to run, code is compiled and deployed to the app.

Flutter’s “Write Once Run All” approach is different, relying on the Skia Graphics Library to render its UI, mimicking native UI components. Just like Xamarin, the logic is centralised in a single Dart codebase.
While we can achieve a “written once” UI that runs perfectly on both platforms, it is recommended not to combine platform-specific UI guidelines in order to prevent UX ‘alienation’ (iOS users are certainly accustomed to their Cupertino-styled apps, while Android users have a more diverse range of styles). In order to dodge this issue, Flutter allows the implementation of different UIs for each platform in a similar way to Xamarin.

The development process isn’t solely based on the cross-platform technology, but also on the whole ecosystem, which includes libraries and plugins. Xamarin is already an established technology, more mature than Flutter, and can call on the support of the most used third party plugins. However, despite the fact that Flutter was only recently developed, it already features an extensive set of third party libraries, just like Xamarin, as well as continuous development tools.

The bottom line of this comparison is that Xamarin emerges as a more consolidated technology, while Flutter, still under significant development, is quickly growing and learning from its competitors how to achieve a solid backbone for the coming months.

3.2. Flutter vs. ReactNative

ReactNative is a cross-platform technology built by Facebook and designed to develop mobile and web apps. Similar to Flutter, ReactNative centralises development in a single codebase and allow developers to write apps for both Android and iOS.

Despite sharing the same codebase, ReactNative embraces differences among platforms and allows platform-specific customisations. ReactNative developers have to rely on third-party UI libraries; this is seen as a disadvantage when compared to the out of the box Flutter UI components.

Facebook built ReactNative as a robust platform, with lots of optimisations. Despite being hard to extract, performance metrics depend on features, algorithms, components amongst many other aspects.

The most relevant consideration to make is the fact that ReactNative apps are developed in JavaScript and, therefore, with a JavaScript engine under the hood, introducing a considerable overhead, as the implementation is not compiled into native code.

ReactNative has been in the market for some years, which translates, in a clear way, to a more mature and consolidated state when compared to Flutter. But besides that, Flutter allows the usage of many JavaScript libraries thus permitting companies to reuse components and to include non-mobile JavaScript developers into mobile projects.

As a result of this ever-moving ecosystem, many third-party libraries were ported from native libraries while others were created specifically for this platform.

4. So far so good. What about real, live, business use cases?

Even though Flutter is still in beta, it is encouraging to note the strong early adoption of the SDK, with some high-profile examples already published. One of the most popular is the companion app to the award-winning Hamilton Broadway musical. Built by Posse Digital, this app has an extensive user base and an average rating of 4.5 on the Play Store (as of July 2018).

Recently, in May 2018, the Chinese e-commerce giant Alibaba announced their adoption of Flutter for Xianyu, one of their flagship apps with over twenty million monthly active users. Alibaba praises Flutter for its consistency across platforms, the ease of generating UI code from designer redlines, and the ease with which their native developers have learned Flutter.

A more complete and organised Flutter portfolio can be found here.

5. Last but not least, what can we conclude?

Based on everything we’ve covered in this article, the conclusion is that Flutter has great potential to solve some of the pains of cross-platform development, but it’s still not clear when it can be a real alternative – this will really depend on the project. It’s not even fair to compare Flutter to Xamarin or ReactNative due to the difference in the maturity of the platforms. However, we think there’s great potential in the technology, and this is why we keep investigating, trying and comparing. With several modern IDEs at its disposal, a stable, developer-friendly modern language and unified output, this technology is bound to earn its place in the digital world.

As soon as Google pushed Flutter through Alpha into a Beta phase, the odds of its success grew exponentially, thus drawing the attention of the mobile community. Mobile experts around the world have found in Flutter what they were looking for as they strived for new, fresh options. Let’s see how it evolves – we’ll definitely be a part of what is to come.

Escrito por:

Pedro Pires

Mobile Developer at Xpand IT

André Baltazar

Developer at Xpand IT

José Camacho

.NET Developer at Xpand IT

Pedro PiresFlutter – A broad Introduction
read more

Sentiment Meter: the perfect mix between Gamification and Artificial Intelligence

The Sentiment Meter is an artificial intelligence (AI) solution conceived, developed and designed by Xpand IT and combines two components: gamification and artificial intelligence. It is a game of emotions in which, after the user fills in a small form, the software randomly selects an emotion that the user has to try to express to the best of their ability. After photographing this moment, the Sentiment Meter evaluates the user’s performance and gives the user a score. In fact, this game has turned out to be a real success, and it can be said that the Sentiment Meter is the perfect mix between gamification and artificial intelligence.

This game was born from the need to create something that would not only show Xpand IT’s technical abilities – ­from our new AI Solutions Centre – but also that would be able to entertain people coming to our stands in the countless events in which we take part. IDC Directions 2018 and Web Summit were the first conferences we took the Sentiment Meter to, and we can say that it did not go unnoticed. Here are some pictures of those moments:

The technology

The logic is very simple: the player spins the wheel of emotions, the computer selects an emotion/facial expression and the player simply has to express that emotion with his/her face. Finally, the interface scores to the player and he/she wins a prize. It seems quite simple; but, actually, what is behind this analysis of emotions is an intelligent algorithm from Microsoft: Azure Cognitive Services. In this case, we use the Face API, which allows processing and recognizing faces and identifies which emotion a person is expressing. This algorithm is fueled by each use and by the pictures taken all over the world.

The whole infrastructure is based on the cloud, in Azure. Some other Microsoft tools were also used, such as SignaIR, which manages interactions in real time between the screen that presents the game and the tablet that gives the commands to the person playing. Moreover, a .NET Core open source framework was used as the basis for this project, allowing for the development of web and cloud apps. The project’s front end was developed on the web with HTML, CSS and Javascript, also relying on some extensions, such as JQuery, Ajax and p5.js.

The team

This whole project would not have been possible without the teamwork of members from our Digital Xperience, UX/UI and Marketing teams, who were able to design and mastermind this solution: Francisco Correia, Senior Project Manager, and Ricardo Duarte, Developer, both from the Digital Xperience department; Marina Mendes, UX/UI Designer, and all the other members of the Marketing team who ensure its proper functioning during events.

 These are the screens of the game:

The Sentiment Meter sets the direction for what it is possible to do today by using artificial intelligence and good ideas born in simple conversations!

Ricardo Duarte, Developer at Xpand IT

With this project we were able to show that integrating Artificial Intelligence services can, even today, set the bar high, whether it concerns interactivity or decision-making.

Francisco Correia, Senior Project Manager at Xpand IT

We were able to bring the user closer to the interface by thinking in emotions and conveying them to technology in a fun and relaxed way.

Marina Mendes, UX/UI Designer at Xpand IT
Ana LamelasSentiment Meter: the perfect mix between Gamification and Artificial Intelligence
read more

Xpand IT’s new Artificial Intelligence Center

The US Merriam-Webster dictionary defines Artificial Intelligence (AI) as:  a field of computer science that works on the simulation of intelligent behaviour in computers; the ability of machines to copy intelligent human behaviour. In fact, this is exactly what Xpand IT seeks to achieve with the new Artificial Intelligence centre: to incorporate an intelligence component in all areas of our society.

AI is a trend that is here to stay, and it will not be long before all companies have incorporated at least one solution simulating human intelligence in their systems, in order to accomplish basic tasks on a daily basis. However, let’s take it easy – we still are not at the level of The Matrix, or Ex Machina! The main focus of Xpand IT is to find real use cases and make  prototype solutions that can ultimately be presented to the end customer. There are plenty of examples of the use of AI that are meant to reduce effort in certain tasks, improve performance and speed in the solving of certain problems, or just gather valuable information that can be used by some departments inside a company. We present a few examples of the use of AI:

Development of a conversational interface (chatbots)

Chatbots are increasingly important in the global technological scene, and it does not take much thought to name several websites that have conversational interfaces trained to help visitors. This type of bot can be a big help to perform simple tasks, such as setting an appointment or buying a movie ticket, and can be applied to countless industries: banking, education, health, retail, and others. The main goal is to have a chatbot that is truly useful for users.

Text analysis and emotion analysis

Currently, we exchange a huge amount of information in text format. Therefore, text analysis ability is expected to improve as the amount of information improves. However, human beings have limits in their ability to process and analyse information, and that is why we have artificial intelligence. By taking advantage of specific techniques and more advanced technologies, it is possible to process all information in record time and to simultaneously gather other types of information, such as the mood of a person who wrote a certain message.

Image or video processing

Another case in which human limits are an opportunity to introduce AI is image (or video) processing. Being able to learn from a large dataset, an artificial intelligence solution can be taught to identify elements in an image or video, a task that would take a lot more time if it was done by human eyes. We can refer to facial recognition for app authentications, or even finding people or specific products in a live video. An AI solution can be an answer to these challenges.

In essence, Xpand IT has gathered specialists from various teams – such as Digital Xperience, Big Data and Data Science – to form a unit of true experts in Artificial Intelligence experiments and solutions, capable of developing projects completely out-of-the-box!

Ana LamelasXpand IT’s new Artificial Intelligence Center
read more