Spectral Multiplexing of QR Codes

Tiago Nuno Pereira Mota

Thesis to obtain the Master of Science Degree in

Electronics and Computer Engineering

Supervisor: Prof. Paulo Sérgio de Brito André

Examination Committee

Chairperson: Prof. José Eduardo Charters Ribeiro da Cunha Sanguino

Supervisor: Prof. Paulo Sérgio de Brito André

Members of the Committee: Prof. Gonçalo Nuno Marmelo Foito Figueira

November 2015

i

Resumo

Com o constante desenvolvimento da utilização de smartphones, um dos principais meios de transmitir

informação usando estes dispositivos, o código QR, sofreu um aumento significativo. Estes códigos de

barras bidimensionais permitem transmitir uma grande quantidade de informação de forma acessível

através de um smartphone.

Nesta dissertação será estudado o aumento de capacidade do código QR, recorrendo a técnicas de

multiplexagem espectral.

Após uma análise inicial do potencial ruido que possa perturbar a leitura dos códigos QR, realizada em

40 códigos Qr diferentes, procedeu-se a utilizar uma técnica multinível para a multiplexagem, utilizando

os bits do código QR para os níveis em escala preta. Esta técnica é ainda utilizada para os espaços de

cor RGB, realizando-se a multiplexagem espectral de códigos QR.

Visto se trabalhar com os espaços de cor RGB em ambas as técnicas mencionadas, a possibilidade de

uma junção de ambas as técnicas enunciadas foi considerada e testada em 40 códigos QR, verificando-

se ser a melhor das técnicas enunciadas para transmissão de elevada quantidade de dados, permitindo

transmitir dados de imagens de dimensões 50x50 e 1,9 KB com facilidade e um ficheiro de som de 23,2

KB com a duração de um minuto e meio com uma relação sinal-ruído muito baixa de 36,03 dB.

Palavras-chave:

códigos QR, multiplexagem espectral, RGB, multinível, binário

ii

iii

Abstract

With the constant development of utilities in smartphones, one of the main means of transferring

information using these devices is QR codes. These bidimensional barcodes allow for a great quantity

of information to be transferred in an accessible way through a smartphone.

In this thesis, a study in order to increase of the data capacity of QR codes will be conducted, using

spectral multiplexing techniques.

After an initial analysis of the potential noise that may disturb the reading of QR codes, the possibility of

using a multilevel technique is considered for multiplexing, by using the bits from the QR codes as the

levels in grayscale, and then for the other RGB color spaces in order to perform spectral multiplexing of

QR codes.

Since both techniques used the RGB color spaces, the possibility for using both techniques at the same

time was considered and testes, demonstrating to be the best of the three, allowing for the data of

images of size 50x50 and 1,9 KB to be transferred with ease and a 23,2 KB audio file to be transferred

with almost no noise in it by having a SNR of 36,03 dB.

Keywords:

QR codes, spectral multiplexing, RGB, multilevel, binary

iv

v

Index

Resumo.................................................................................................................................................... i

Abstract .................................................................................................................................................. iii

Index ........................................................................................................................................................v

Figures list ............................................................................................................................................ vii

Tables list ............................................................................................................................................... ix

Acronyms list ......................................................................................................................................... xi

1. Introduction ........................................................................................................................................ 1

1.1 Motivation ......................................................................................................................................... 1

1.2 State of the art ................................................................................................................................. 3

2. Theory ................................................................................................................................................. 7

2.1 The QR code .................................................................................................................................... 7

2.2 Noise in images ............................................................................................................................. 16

2.3 Color spaces .................................................................................................................................. 18

3. Implementation ................................................................................................................................ 21

3.1 Noise ............................................................................................................................................... 21

3.2 Multilevel quick response codes ................................................................................................. 24

3.3 Multiplexing with color coding ..................................................................................................... 29

3.4 Multilevel color spectral multiplexing ......................................................................................... 33

4. Large capacity QR Code ................................................................................................................. 35

4.1 Image transfer ................................................................................................................................ 35

4.2 Audio transfer ................................................................................................................................ 39

5. Conclusions ..................................................................................................................................... 43

References ........................................................................................................................................... 45

vi

vii

Figures list

Fig. 1 Example of QR code being used in advertisement ....................................................................... 1

Fig. 2 Examples of an UPC-A barcode and a barcode in patient wristband ........................................... 2

Fig. 3 Example of a 3D code based of on a QR code ............................................................................. 3

Fig. 4 UPC barcode and a EAN8 barcode .............................................................................................. 4

Fig. 5 A typical Code 39 barcode ............................................................................................................ 5

Fig. 6 Data matrix barcode in its square form ......................................................................................... 5

Fig. 7 Example of an Aztec barcode ....................................................................................................... 6

Fig. 8 QR code encoding process ........................................................................................................... 8

Fig. 9 Pattern positions in a QR code .................................................................................................... 12

Fig. 10 Data bit placement in a QR code .............................................................................................. 13

Fig. 11 QR code decoding process ....................................................................................................... 16

Fig. 12 Gaussian noise in a QR code produced in Matlab for variance = 0,1 ....................................... 17

Fig. 13 Salt-and-pepper noise in a QR code produced in Matlab for noise density = 0,2 ..................... 17

Fig. 14 Shot noise in a QR code produced in Matlab ............................................................................ 18

Fig. 15 RGB color spaces...................................................................................................................... 18

Fig. 16 Three dimensional representation of RGB color space ............................................................ 19

Fig. 17 Digital image color flow [11] ...................................................................................................... 19

Fig. 18 Speckle noise in QR codes for 250 characters for L error correction level and M error

correction with variance 0.04, 0.2 and 0.3 ......................................................................................... 22

Fig. 19 Shot noise (Poisson) in QR codes for 250 characters for L and M error correction levels ....... 23

Fig. 20 Salt and Pepper noise for 250 characters for an L error correction level with noise density of

0.1, 0.2 and 0.7 respectively ............................................................................................................. 23

Fig. 21 Zero Mean White Gaussian noise with 0.01 variance for 250 and an L and M error correction

levels for 250 characters ................................................................................................................... 23

Fig. 22 Multilevel intensities for a grayscale QR code [10] ................................................................... 24

Fig. 23 Multilevel intensities for color channels in the conventional QR code [10] ............................... 24

viii

Fig. 24 Example of a colored QR code using the method described [10] ............................................. 25

Fig. 25 Examples for colored QR codes using a single color for all the codes ..................................... 26

Fig. 26 QR codes in blue color space ................................................................................................... 28

Fig. 27 QR codes in blue color space after being multiplied by their respective values ....................... 28

Fig. 28 "Adding” of previous QR codes ................................................................................................. 28

Fig. 29 Multiplexing and color coding table for 3 QR codes [3] ............................................................. 30

Fig. 30 Examples of multiplexing 2 QR codes for different color spaces .............................................. 31

Fig. 31 QR color code using the first code for red and blue and second code for green ...................... 31

Fig. 32 Multiplexing of 3 QR codes into one where each code has a different color space ................. 32

Fig. 33 Example of multiplexing 6, 9 and 12 different codes into one ................................................... 33

Fig. 34 50x50 image and QR code holding its information ................................................................... 36

Fig. 35 Version 18 QR code holding image information ........................................................................ 37

Fig. 36 Version 40 QR code holding data file information ..................................................................... 40

Fig. 37 Audio file and reconstructed audio file ...................................................................................... 41

ix

Tables list

Table 1 Data capacity (characters) for different versions of QR codes .................................................. 7

Table. 2 Binary indicator for existing modes ........................................................................................... 9

Table 3 Number of bits required by each mode and version .................................................................. 9

Table. 4 Masking patterns for QR codes ............................................................................................... 14

Table. 5 Bits associated with each Error Correction Level .................................................................... 15

Table. 6 Letter frequency for the english language ............................................................................... 21

Table 7 Average values for when the QR code becomes unreadable .................................................. 22

Table 8 QR code values for amount of codes being multiplexed .......................................................... 27

Table 9 Possible color combinations for a color QR code .................................................................... 29

Table 10 RGB values for 3 QR codes being multiplexed ...................................................................... 38

x

xi

Acronyms list

AIAG - Automotive Industry Action Group

CCD – Charge Coupled Device

EIA - Electronic Industries Alliance

EC – Error Correction

EAN – European Article Numbering

ICC – International Color Consortium

CIE - International Commission on Illumination

OCR – Optical Character Recognition

1D – one dimensional

QR – Quick Response

RGB – Red Green and Blue

TIFF/EP - Tagged Image File Format / Electronic Photography

2D – two dimensional

UPC – Universal Product Code

xii

1

1. Introduction

1.1 Motivation

With the growth in today’s society, where almost anyone has a smartphone with an internet connection,

companies strive to outdo each other and find a way to gain consumers. One of the ways they came up

with, was by taking advantage of the smartphones apps and camera to act as a barcode reader. With

this technology, companies started using QR codes in posters to help divulge their products or as

promotional giveaways much like the food stamps you would get by visiting fast food places like

Burguerking or Pans, and so the popularity of these codes skyrocketed.

Fig. 1 Example of QR code being used in advertisement

But even with the high capacity to hold data that these codes provide, its true potential has not yet been

reached and researchers strive to develop better QR codes that can hold even greater amounts of data

by using several methods that will be explored in this thesis.

But it’s not just advertisement companies that have an interest in barcodes. Due to the versatility of

barcode technology, one dimensional codes have a wide use in society, such as in the case of

manufacturing companies or the healthcare industry. In the case of the manufacturing industry, barcodes

will provide real-time information on products thus improving product stock management by being able

to identify and track products without the need to access a database, or by being to perform quality

control and testing on said products [1, 2].

In healthcare, error and resultant malpractice minimization control can be done by monitoring which

medicines and equipment was used on the patient, and by whom, where and when it was used, by

having a barcode with said information on the patients wristband as seen in fig.2 . Barcodes also allow

for inventory control and replenishment, traceability systems and faster and more accurate billing [2].

2 Fig.2 retrieved from https://upload.wikimedia.org/wikipedia/commons/5/5d/UPC-A-036000291452.png and https://upload.wikimedia.org/wikipedia/en/a/ad/LB2-ADULT-L3_Assembled.jpg

Fig. 2 Examples of an UPC-A barcode and a barcode in patient wristband

With the advance in today’s technology, their use has expanded greatly into the publicity area by

upgrading from 1D codes to 2D codes such as QR codes.

These barcodes were developed by the Denso Wave Incorporated in 1994 as a means to transmit

information through a print-scan channel for the automotive industry and were approved as an ISO

international standard (ISO/IEC18004) in June 2000 [1, 2, 3].

In recent years their use has expanded to product tracking, item identification, time tracking, document

management as well as general marketing, due to its simplicity and compatibility with smartphones [4].

Driven by the development in camera phones, these smartphones can work as scanners, barcode

readers and portable data storages with network connectivity, which when used with a 2D barcode allow

it to work as a bridge between the physical world and the digital world, such as allowing consumers to

purchase online e-tickets and simply using them at the counter instead of waiting in lines by having the

QR code on their phone that contains their ticket information read by a scanner [2].

It is clear that QR codes have a great uses be it in healthcare, the manufacturing industry or even the

advertisement industry, but are limited by their capacity. Therefore the objective of this work is to study

how to implement spectral multiplexing techniques such as multilevel or color coding, in order to

increase the capacity that the QR codes currently possess and demonstrate that with this increase in

capacity that the QR codes could potentially transmit other kinds of data than just text, such as images

or even audio files, which could open up new possibilities for the advertisement or even the music

industry.

3 Fig.3 retrieved from http://www.mobiliodevelopment.com/wp-content/uploads/2012/03/3D-QR.jpg

1.2 State of the art

Barcodes are considered a machine-readable representation of information formed by the combinations

of high and low reflectance regions at the surface of an object, which can be converted into a binary

representation. In the initial version, information was encoded into an array of adjacent bars and spaces

with varying width, thus being called “barcodes”. The first barcodes designed were the one-dimensional

ones, where the code was read by a scanner that swept the code with a beam of light in a straight line

[2].

By switching the bars and spaces with dots and spaces arranged in an array or matrix, we can increase

the density of data within space, resulting in a two-dimensional barcode. Obviously this means that

unlike the 1D code, the 2D ones will require a scanner that can read vertical and horizontal directions.

We can find also a three-dimensional coding that is often used in irregular surfaces. The code uses the

‘bars’ from the 1D codes or the ‘dots’ of the 2D codes and embosses them on the surface of the material.

The 3D barcodes use special elements to generate high and low reflectance instead of a visual contrast

between different colors as seen in fig.3 [2]. This method would allow a further increase in the capacity

of the previous codes since for different layers of thickness we could possibly transmit additional

information. It is to note that mobile scanners such as the ones in telephones do not currently have the

capacity to read these codes.

Fig. 3 Example of a 3D code based of on a QR code

Being one of the several types of automatic identification and data capture systems used nowadays,

alongside RFID systems, optical character recognition (OCR) Systems and magnetic stripe technology,

barcode technology was originally developed around the 1800s being used at first as a reading aid for

the blind [2].

Around the 1950s thanks to the technological advancement in computing, data recognition patterns

were developed, where a patent by the name of ‘Classifying apparatus and method’ was filed by Norman

4 Fig.4 retried from https://www.oclc.org/content/dam/oclc/bibformats/images/upc_code.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/f/f1/EAN8.svg/738px-EAN8.svg.png

J. Woodland and Bernard Silver composed by a series of concentric circles instead of in straight lines

used for 1D codes [2].

This patent was the first step towards what we have as the current linear barcodes, and at that time,

one of its creators, Woodland, thought of a barcode using vertical lines instead of circles, but decided to

use the concentric circles since it allow omnidirectional scanning [2].

Nevertheless, the early stages of barcode symbology was difficult due to inadequate pattern reading

and scanning technology, since only transducers were capable of scanning patterns at the time, unlike

the ones we have currently that use pen and flashlight-type light sources and mirrors, but thanks to the

inventions of the moving-beam laser scanner in the 1960s, and of the charge coupled device (CCD)

scanner and pixel readers, further development of barcode systems became possible.

The first adopted linear barcode was called Universal Product Code (UPC), where it was considered

that all the interest groups would use the same symbol structure, allowing for each code to be uniquely

identified [2].

Due to foreign interest by the European Article Numbering Association in UPC, it was adopted as the

European Article Numbering (EAN) code format as seen in fig.4. This code could be omnidirectionally

read, was compatible with UPC, did not require special printing and wrapping and could be decoded at

a speed within 100 inches per second [2].

Fig. 4 UPC barcode and a EAN8 barcode

The difference between these 2 formats was that while the UPC was considered for use only in the

United States, the EAN format was considered for a global use and so it includes two or three

identification numbers as a prefix that identifies the GS1 Member Organization. The EAN format was

adopted by more than a hundred countries and so is considered the global standard one dimensional

barcode.

The military of the USA adopted the barcode ‘Code 39’, seen in fig.5, for logistics applications of

automated marking and reading symbols (LOMARS),due to its ability to encode alphanumeric data it

became one of the most used industrial barcodes, being mandated by the Automotive Industry Action

Group (AIAG) and the Electronic Industries Alliance (EIA) for labelling [2].

5 Fig.5 retrieved from http://help.accusoft.com/BarcodeXpress/v9.0/activex/images/code39.fpg Fig.6 retrieved from http//www.roemerind.com/wp-content/uploads/2010/07/Data-Matrix_Roemerind.jpg

Fig. 5 A typical Code 39 barcode

Since the 1D barcodes were introduced, there has been a growing demand to encode more and more

data than what a 1D barcode could accommodate. The Toyota Motor Corporation, along with the Denso

Wave Inc. tried to devise a barcode symbol that was similar to the 2D barcodes, and proposed a code

called the Codabar [1, 2, 3].

The Codabar was used in a parts-movement system called Kanban, which meant a plate or a card with

necessary information for the production, in order to increase the efficiency of the management of parts

flowing along the production line.

In 1978 a project utilizing Kanban and 2D barcode like symbologies began, which lead to the

development of current societies 2D barcodes that could encode from 10 to 100 times more data than

their 1D counterparts, while also requiring less space when an identical set of data is encoded thanks

to their matrix format.

One of such 2D barcodes is the Data Matrix barcode as seen on the right of fig.6. Developed by RVSI

Acuity Cimatrix, this barcode ranges from an 8x8 size to a 132x132 size, capable of holding up to 2335

characters in alphanumeric or 3116 numbers, and 71 characters and 96 numbers in its rectangle form.

Another often used 2D barcode is the Aztec code. Invented by Andrew Longacre, this 2D barcode has

the potential to use less space than other matrix barcodes due to not requiring a blank ‘quiet zone’

around it. It ranges from 15x15 modules up to 151x151 by adding several layers of 2 pixels around the

core. This code can encode up to 3067 characters and 3832 digits.

Fig, 6 Data matrix barcode in its square form

6 Fig.7 retrieved from http://www.mobiliodevelopment.com/wp-content/uploads/2013/06/aztec-code-2d-matrix-barcode.gif

As said before, 2D barcodes have many uses in healthcare, such as allowing elderly people to better

medicate themselves by giving them verbal instructions with the help of a voice app [5] and electronics

industries due to the need of handling small products [6], but that does not mean its usage is only in

those areas, as demonstrated by the automotive industry or even the security industry [7]

With the increase of capacity provided by 2D barcodes, they can now provide error detection and

correction capability, multilingual encoding (such as byte, alphanumeric, kanji) and different symbol

versions depending on the data required, allowing it to work as portable database, unlike the 1D

barcodes that work as an index to a backend database [2].

Fig 7 Example of an Aztec barcode

7

2. Theory

2.1 The QR code

The QR code is composed by geometric patterns of black and white dots (modules), in a square white

background. The code has 3 finder patterns in the upper left, bottom left and top right corners that are

made from three overlapped homocentric squares of 7x7 black colored modules, 5x5 white colored

modules and 3x3 black colored modules [1, 3]. These finder patterns are used to define the position of

the seeking graph and obtain the image information from the reader.

The QR code also enables four type of encoding modes (numeric, alphanumeric, byte/binary and kanji)

which vary in capacity depending on the level of error correction and the version of the code.

As mentioned before, there are four levels of error correction by using Reed-Solomon error correction,

L (low), M (medium), Q (quartile), H (high) where 7%, 15%, 25% and 30% of symbols can be corrected

in case the original code has been damaged [1, 3]. This process creates error correction codewords

based on the data, which can be used by the reader to determine if the data was read correctly and if

so, use those same codewords to correct those errors.

Table 1 Data capacity (characters) for numeric, alphanumeric and kanji modes for different versions of QR codes

Error Data Capacity

Version Correction Numeric Alphanumeric Kanji Level

1 L 41 25 10

M 34 20 9

H 27 16 7

Q 17 10 4

5 L 255 154 85

M 202 122 52

H 144 87 37

Q 106 64 27

6 L 322 154 65

M 255 12 52

H 144 87 37

Q 106 64 27

7 L 370 224 95

M 293 178 75

H 207 125 53

Q 154 93 39

40 L 7089 4296 1817

M 5596 3391 1435

H 3993 2420 1024

Q 3057 1852 784

8

There are forty different versions for QR codes, going from 21x21 to 177x177 modules. The readers in

actual cellphones usually can only read up to the tenth version. These versions differ by stepping four

modules horizontally and vertically.

The encoding/decoding process is regulated by the norm ISO/IEC18004:2006 and is described in a

detailed way in the on-line website http://www.thonky.com/qr-code-tutorial/introduction.

The process of encoding of the QR code can be divided into six steps:

1. Data analysis, where the encoding mode most suited for the characters will be chosen;

2. Coding, where the Error Correction Level (ECL) will be chosen before the actual coding of the

message to its bit equivalent is performed so that the adequate QR code version will be used;

3. Error Correction, where the structure message will be divided into datablocks and where error

correction bits will be generated;

4. Arrangement, where the several patterns, modules, separators and the encoded message will

be placed in the QR code matrix;

5. Masking, where the QR code matrix will be changed to 8 different versions and submitted to a

scoring test to see which version is the best one;

6. Format information, where the error correction level and masking chosen will be placed in the

QR code matrix along with the QR code version.

Fig. 8 QR code encoding process

As was said, in the data analysis step the first thing to do is chose from one of four modes to encode

the message. The numeric mode as the name implies is used for decimal digits, that is “0,1,2,3,…,” ,

while alphanumeric mode also includes uppercase letters, symbols and a space. Byte mode is used if

the characters are not in the alphanumeric mode and Kanji mode is used for any characters belonging

to the Shift JIS character set. It is to note that while Shift JIS characters can be represented in UTF-8, it

can be possible to use byte mode instead of kanji mode to encode, but this requires three or even four

bytes while Shift JIS can be encoded in one or two bytes, thus kanji mode offers the highest capacity

for it. It is possible to use multiple modes in a single QR code by specifying the mode before the section

of bytes that uses said mode. QR codes also have difficulty accepting characters with punctuation such

as ’ã’ since these are not included in either alphanumeric or byte tables.

9

Before proceeding to the coding, one of the error correction levels must be selected, since a higher error

correction level will mean the QR code will have to be larger. After this is done, the version for the QR

code will be chosen based on the number of characters to be encoded and the error correction level

chosen.

We now need to add the mode indicator that identifies the mode select since the encoded data must

start with it in order to specify the mode being used for the bits. These mode indicators are presented in

the table 2.

Table 2 Binary indicator for existing modes

Mode Name Mode Indicator

Numeric Mode 0001

Alphanumeric Mode

0010

Byte Mode 0100

Kanji Mode 1000

A character count indicator will need to be added as well that will be placed after the mode indicator.

This indicator will essentially represent the number of characters being encoded and will have a specific

size depending on the QR version as shown below.

Table 3 Number of bits required by each mode and version

Mode Version

1 - 9 10 - 26 27 - 40

Alphanumeric 10 12 14

Numeric 9 11 13

Byte 8 16 16

Kanji 8 10 12

Moving on to the encoding step, the message will need to be turned into its binary equivalent depending

on the encoding mode chosen. If the numeric mode was chosen, the digits will be split up into groups

of three and then convert them into binary. If it is alphanumeric, the characters will be split into groups

10

of two, converted into their number representation from the alphanumeric table, the first character will

be multiplied by 45 and then the second character will be added and the resulting sum will be converted

into binary. If byte mode is used, the characters will be converted into hexadecimal bytes and then into

an 8-bit string, padding it them on the left with 0’s to make it 8 bit long if necessary. If it is kanji the same

process as byte mode occurs but with the length of 13 bits instead of 8.

Each version and error correction level has a total number of bits required. In case the length required

wasn’t achieved after coding the message, the following steps will be performed until the length is

achieved:

add a terminator of up to four 0 bits at the right side of the string depending on how many bits

are left to reach the max length;

add 0 bits at the end of the string until it is a multiple of 8;

add 11101100 00010001 at the end of the string, repeating this final step until it reaches the

maximum length.

The error correction step begins by breaking the data codewords into groups and blocks depending on

the version and level chosen if necessary. This process begins by grouping the string into 8 bit parts

and converting then back into decimal. These numbers will them be used as the coefficients of the

message polynomial. After this is done, the generator polynomial will be chosen based on the version

and error correction level of the code, so that we can divide the message polynomial by it. This division

is based on the polynomial long division, considering the Galois field arithmetic.

The way polynomial long division works is:

1. take the divisor and multiply it by something that can make its first term equal to that of the

polynomial of the dividend;

2. subtract the new polynomial by the dividend;

3. if it is not possible for the remainder to be multiplied by an integer, the operation is complete,

else the process will repeat itself by using the remainder as the new dividend until this becomes

true.

Below is a very simple example of how a polynomial long division looks like by choosing 𝑥2 + 𝑥 − 1 as

the dividend and 𝑥 + 1 as the divisor:

1. multiply 𝑥 + 1 by 𝑥, resulting in 𝑥2 + 𝑥;

2. subtract 𝑥2 + 𝑥 from 𝑥2 + 𝑥 − 1, which leads to a remainder of -1;

3. since the remainder can no longer be multiplied by an integer the operation is complete.

It is necessary to note that the polynomial long term division is simpler in error correction since there is

no need to deal with the exponents of the polynomial terms.

11

Regarding the Galois Field, in QR code the bit-wise modulo 2 arithmetic and byte-wise modulo

100011101 arithmetic are used, which means we use Galois Field 28, also known as Galois Field 256.

The numbers contained in the Galois Field 256 range from 0 to 255, which is also the range of numbers

that an eight-bit byte covers, allowing for the mathematical operations performed in the Galois Field 256

to be represented as eight-bit bytes.

Negative numbers have the same values as positive numbers when in the Galois Field, so the absolute

value of the numbers are used when dealing with Galois Field arithmetic. Operations in the Galois Field

are cyclical, which means that if a result is higher than 255, it will require a modulo operation to get a

number that is in the Galois Field. Another trait that the absolute values of numbers have, is that

subtracting or adding is the same thing thanks to the modulo 2 operation performed afterwards which is

the equivalent of performing a XOR operation.

Returning to the byte-wise modulo 100011101, the QR code specifies that whenever a number is larger

than 256 it should be XORed with 285 that is, the decimal equivalent of 100011101, in order to get a

lower number than 256. This allows for all the numbers in the Galois Field 256 to be represented with

2n.

As said before in the polynomial long term division, 2 polynomials are required, which in this case will

be the message polynomial and the generator polynomial. The message polynomial uses the data

codewords obtained in the data encoding as the polynomial’s coefficients, while the generator

polynomial is created by multiplying(𝑥 − 20)… (𝑥 − 2𝑛−1), where n is the number of error correction

codewords that need to be generated.

After this is done begin the division steps below will be performed:

find the appropriate term to multiply the generator polynomial. The result should have the same

first term of the message polynomial or the remainder;

Apply XOR to the result of message polynomial or the remainder;

Repeat these steps x times depending on the x number of data codewords.

After the error correction codewords are found, it may be necessary to break up larger QR codes into

blocks that vary depending on version and error correction level. At this point the blocks will be

interleaved by taking the first data codeword from the first block, then the first from the second block

until it reaches the last block, where it returns to the first block and takes the second codeword and

repeats the process, until the last data codeword from the last block at which point the process is

repeated but for the error correction codewords.

The final message can now be convert back into binary in order to put the bits in the matrix for the

arrangement step. It is to note that some versions need additional bits called remainder bits, at the end

of the string after converting the final message back into binary.

12

Fig. 9 Pattern positions in a QR code

Like it was said before the finder patterns are positioned at the top left, top right and bottom left corner

of the QR code and consist of an outer black square of 7 by 7 modules, an inner with square of 5 by 5

modules, and at last a black square in the center of 3 by 3 modules. This pattern was designed to be

unlikely to appear in any other sections of the QR code thus allowing the scanners to detect them and

orient the code for decoding.

Since the size of the code can be calculated by the formula (𝑉 − 1) ∗ 4 + 21, where V is the version of

the code, the top left finder pattern’s top left corner is always placed at (0,0), the top right finder pattern’s

top left corner is always placed at ((𝑉 − 1) ∗ 4 + 21 − 7, 0) and the bottom left finder pattern’s top left

corner is placed at (0, (𝑉 − 1) ∗ 4 + 21 − 7).

After adding the finder patterns, the separators can be added as well, which are lines of white modules

of only one module wide that are placed beside the finder patterns in order to separate them from the

rest of the code.

The alignment patterns are used on versions above version 1, and consist of a 5 by 5 black module

square, an inner 3 by 3 white module square and a single black module in the center. The locations of

these alignment patters are defined in the alignment pattern locations table where the numbers to be

used indicate the row and column coordinates for the center black module. It is to note that these

patterns cannot be put into the matrix before the finder patterns and the separators and they must not

overlap with any of them.

Next the timing patterns are added, which are two lines, one horizontal and one vertical of alternating

black and white modules. The horizontal timing pattern is placed in the 6th row of the code between the

separators and the vertical timing pattern is placed on the 6th column of the code between the separators.

These patterns always start and end with a black module and may overlap with the alignment patterns

since the white and black modules always coincide.

Alignment Patterns

Finder Patterns

Timing Patterns

Separators

Dark Module

13

Before the data bits are added, the dark module and the reserved areas for the format and version will

need to be placed as well. The dark module will be placed beside the bottom left finder pattern at the

coordinate (4 ∗ 𝑉 + 9 , 8), while the reserved areas for the format area will be placed one module strip

bellow and to the right of the separator near the top left finder pattern, a one module strip below the

separator of the top right finder pattern, and a one module strip to the right of the separator near the

bottom left finder pattern. The reserved version information area required for codes with version 7 or

higher which are 6 by 3 block will be placed above the bottom left finder pattern and a 3 by 6 block will

be placed to the left of the top right finder pattern.

With that done the data bits can finally be placed. The way this works is starting from the bottom right

of the matrix, we proceed upward in a column that is 2 modules wide, by placing the first bit on the right

side and the second one on its left and then going up a line and repeating this process. Once the top is

reached, we move to the next 2 module column to the left of the previous one and instead continue

downward until the bottom where the upward placement begins again so on and so forth. If a function

pattern or a reserved area is encountered the placement continues on the next unused module.

Fig.10 Data bit placement in a QR code

After all the bits are placed in the matrix, the masking step will commence. Currently there are eight

different mask patterns with different specifications where if the module is masked it will change from

black to white or vice-versa. The mask patterns should only be applied to data modules and error

correction modules.

14

Table 4 Masking patterns for QR codes

Mask Number

If the formula below is true for a given row/column coordinate, switch the bit at that coordinate

0 (row + column) mod 2 == 0

1 (row) mod 2 == 0

2 (column) mod 3 == 0

3 (row + column) mod 3 == 0

4 ( floor(row / 2) + floor(column / 3) ) mod 2 == 0

5 ((row * column) mod 2) + ((row * column) mod 3) == 0

6 ( ((row * column) mod 2) + ((row * column) mod 3) ) mod 2 == 0

7 ( ((row + column) mod 2) + ((row * column) mod 3) ) mod 2 == 0

When applied to the matrix, each mask pattern has a penalty score based on four rules. The encoder

must test all of the mask patterns and the one which receives the lowest penalty is the one used in the

final output. The way the evaluation is conducted is based on the whole matrix, even though the masking

is only applied to data and error correction modules with the reserved areas considered to be white

modules.

The penalty rules are then as follows:

The first rule gives the QR code a penalty for each group of five or more same-colored modules

in a row (or column).

The second rule gives the QR code a penalty for each 2x2 area of same-colored modules in the

matrix.

The third rule gives the QR code a large penalty if there are patterns that look similar to the

finder patterns.

The fourth rule gives the QR code a penalty if more than half of the modules are dark or light,

with a larger penalty for a larger difference.

For the first rule, if 5 consecutive modules are of the same color, add 3 to the penalty and 1 for each

consecutive one and at the end total the horizontal and vertical results.

15

Regarding the second rule, the person evaluating should look at areas that are 2x2 or larger and add 3

for each possible 2x2 block.

The third rule occurs when a set of black-white-black-black-black-white-black modules followed by 4

white modules on either side occurs that gives a 40 penalty score.

The last rule is based on the ratio of white and dark modules. The total number of the modules in the

matrix counted and the total number of black modules in it as well and the percentage for them is

calculated. For the next step of this rule, the previous and next multiple of five of this percentage must

be found and subtracted by 50 for each of them and take the final absolute value. For the final step of

this rule, each of the previous values are divided by five and then multiplied by the lower of the two by

10 which will give the final penalty score for this rule.

With all of the penalties measured the results for each of the mask patterns can be compared and the

one with the lowest score will be the one chosen to be applied to the QR code.

With the masking pattern done we arrive at the last step of creating the code: the format and version

information strings.

There exists 32 possible format information strings (8 mask patterns times 4 error correction levels) and

it is always 15 bits long. In order to create it, a five bit string is required that encodes the error correction

level and the mask pattern used in the code, which will generate ten error correction bits. These 15 bits

are XORed with the mask pattern 101010000010010 and afterwards the format string will be placed into

the QR code in the areas reserved for them in the arrangement step.

If the version of the code is 7 or larger a version information string with the length of 18 bits will have to

be generated. Six of these bits encode the version while the remaining twelve are error correction bits.

After generating the string the bits are inserted into their designated areas and the final matrix is

obtained.

Table 5 Bits associated with each Error Correction Level

Error Correction

Level Bits

Integer Equivalent

L 01 1

M 00 0

Q 11 3

H 10 2

16

When a QR code is received by a reader, the decoder compensates the geometrical transformation of

the code. The modules are recognized to remove the version information, the information on the used

mask pattern and the error correction level. The QR code is then unmasked to extract the message bits

and the error correction bits. The bits are arranged in blocks depending on the version of the QR code

and its error correction level and are then decoded to restore the original data [8].

Fig. 11 QR code decoding process

2.2 Noise in images

Even though QR codes are mostly known for the resilience to noise interference, it can still happen

when the reader from the decoder detects an image, from the reader being overheated, or the light

conditions in which the image is acquired are not ideal.

One of the possible noises that can affect an image is Gaussian noise. Gaussian noise occurs mostly

in digital images during the acquisition by sensors with poor illumination and/or high temperature and it

will be more dominant in the darker areas of the image [9].

The probability density function for Gaussian Noise is as follows where z is the grey level, µ the mean

value and α the mean value:

𝑝𝐺(𝑧) = 1

𝛼√2𝜋𝑒−(𝑧−𝜇)2

2𝛼2

17

Fig. 12 Gaussian noise in a QR code produced in Matlab for variance = 0,1

Salt-and-pepper noise originates mostly from analog-to-digital converter errors or bit errors in

transmission. The image affected by it will have dark pixels in bright regions and bright pixels in dark

regions and it can be mostly eliminated by using dark frame subtraction.

𝑝𝐼(𝑧) = {𝑃𝑎 𝑓𝑜𝑟 𝑧 = 𝑎𝑃𝑏 𝑓𝑜𝑟 𝑧 = 𝑏0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

Fig. 13 Salt-and-pepper noise in a QR code produced in Matlab for noise density = 0,2

Shot noise occurs from variations of the number of photons sensed at an exposure level thus also being

known as photon shot noise. Shot noise follows a Poisson distribution that approximates to a Gaussian

distribution at high levels.

𝑃(𝑋 = 𝑘) = 𝜆𝑘𝑒−𝜆

𝑘!

18 Fig. 15 retrieved from https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/CIE1931xy_CIERGB.svg/495px-CIE1931xy_CIERGB.svg.png

Fig. 14 Shot noise in a QR code produced in Matlab

Quantization noise as the name implies is caused by quantizing the pixels of a sensed image to a

number of discrete levels and it has approximately a uniform distribution.

2.3 Color spaces

RGB bases were developed in order to be used as interchangeable spaces to communicate color and/or

as working spaces in imaging applications. These color spaces ease the process of color communication

in the way that a user or device can perceive and understand the color of the image and alter it if

necessary. If applied correctly, the RGB color space will minimize color space conversions in an imaging

workflow, improving image reproducibility and facilitating accountability [10, 11].

Several RGB color spaces were developed as seen in Fig. 15, since there is no consensus on a single

RGB color space that satisfies all imaging needs.

Fig. 15 RGB color spaces

The International Commission on Illumination defined color space as a geometric representation of

colors in space usually of three dimensions as seen in Fig. 16.

19

Fig. 16 Three dimensional representation of RGB color space

The base functions used are color matching functions called CIE floor matching functions. Color spaces

are considered a subset of spectral spaces, which in turn are spaces spanned by a set of spectral basis

functions.

A color flow of a digital image is explained as follows: first an image is captured into a sensor or source

device space. This image may be altered into an unrendered image space (2 standard color space that

describes the original colorimetry) but in most cases it is usually directly transformed from the source

device space into a rendered image space that describes the color space as a real or virtual output [10].

In case that rendered image space describes a virtual output, it is necessary additional transforms to

convert the image into output space that is an output device specific color space.

Most of standard RGB color spaces are under the category of rendered image spaces.

Fig. 17 Digital image color flow [11]

20

When a scene is acquired by a scanner or a digital camera, the first representation of the color space is

device and scene specific depending on the illumination, the sensor and the filters.

When images are archived or communicated in sensor space, the characterization data has to be

maintained so that further color and image processing is possible, and so images should be saved in

TIFF/EP that has defined tags to hold the information.

Depending on the image and the device, the transformation from sensor space to unrendered space

requires linearization, pixel reconstruction, white point selection and a matrix conversion.

The unrendered image color space gives us an estimate of the original colorimetry while maintaining

the relative dynamic range and gamut of the original and if encoded in higher bit-depth will have the

advantage of being processed in tone and color for different purposes and devices with the quality of

the colorimetric estimate depending on the ability to choose the correct scene adopted white point and

transformations.

It is to note that unrendered images need to be placed under additional transforms to make them

viewable or printable and so appearance modeling can be applied when a reproduction is needed.

When color spaces are based on the colorimetry of real or virtual output characteristics they are called

rendered image spaces. These can be obtained by transforming source or unrendered image spaces

as stated before, with the complexity of the transforms varying from video-based approach to image

dependent algorithms [11]. These transforms are mostly non-reversible as some information of the

original scene is lost in the encoding or in the compression to fit the dynamic range and gamut of the

output.

The rendered image spaces and designed to resemble the output device characteristics, to ensure that

minimal loss occurs when converting to the output specific space.

Following the trend of rendered spaces, the transforms to output spaces are device and media specific

as well. If the rendered space is equal or close to the device characteristics, transformations aren’t

necessary, otherwise additional conversions are needed and accomplished by using ICC color

management workflow.

In case the gamut, dynamic range and viewing conditions of the rendered space are very different from

the output space, it is advised to use a reproduction model that allows image specific transforms than

to use the color management system mentioned [10, 11].

21

3. Implementation

3.1 Noise

Now that the basis for the thesis has been laid down, the techniques for spectral multiplexing can be

tested. These techniques can only be applied on same version QR codes since each version has a

specific size and different versions would overlay incorrectly.

The document used as basis for this simulation and further ones along the line in this thesis was “To

Increase Data Capacity of QR Code Using Multiplexing with Color Coding: An example of Embedding

Speech Signal in QR Code” and the website used to transform it into a QR code was

http://www.racoindustries.com/barcodegenerator/2d/qr-code.aspx.

This document follows the word frequency established for the English language very closely, differing

only.in this document for the letter ‘c’ and ‘h’ where they differ for more than 2% in occurrence. Letter

frequency is mostly used as a basis for encoding methods, one of the most commonly known being

Morse code, and as a way to identify languages since each language has a certain distribution attached

to it

Table 6 Letter frequency for the english language

Before starting to experiment on the QR code, an analysis on the effects of noise was conducted on

several versions of QR codes to see the threshold from when a QR code became unreadable.

A sample size of 40 different images, 4 for each version of the QR codes from version 1 to version 10

and the error correction levels. Each of the images was put under test 5 different times for each of the

types of noise and the average of the results was presented in the table below.

22

Table 7 Average values for when the QR code becomes unreadable

Luckily, Matlab already has tools in place that simulates different types of noise, so it was only necessary

to choose values for variance or in the case of Salt $ Pepper, noise density.

From this experiment it was observable that there isn’t one specific level of noise in which the image is

unreadable since obviously noise doesn’t present itself in the same patterns every time and thus there

is “average” level in which the images do in fact become unreadable, except for Poisson noise where it

didn’t affect any of the QR codes in any of the error correction levels.

L M Q H L M Q H L M Q H L M Q H

1 0,2 0,2 0,1 0,1 R R R R 0,2 0,1 0,1 0,2 0,1 0,1 0,1 0,1

2 0,2 0,2 0,1 0,2 R R R R 0,1 0,1 0,2 0,2 0,1 0,1 0,1 0,1

3 0,2 0,2 0,2 0,1 R R R R 0,1 0,1 0,2 0,1 0,1 0,1 0,1 0,1

4 0,2 0,1 0,1 0,2 R R R R 0,2 0,1 0,2 0,2 0,1 0,1 0,1 0,1

5 0,1 0,2 0,2 0,2 R R R R 0,2 0,2 0,2 0,2 0,1 0,1 0,1 0,1

6 0,2 0,1 0,1 0,1 R R R R 0,1 0,1 0,2 0,1 0,1 0,1 0,1 0,1

7 0,1 0,1 0,1 0,2 R R R R 0,1 0,2 0,1 0,1 0,1 0,1 0,1 0,1

8 0,1 0,1 0,2 0,1 R R R R 0,1 0,1 0,2 0,2 0,1 0,1 0,1 0,1

9 0,1 0,1 0,1 0,1 R R R R 0,1 0,1 0,1 0,1 0,1 0,1 0,1 0,1

10 0,1 0,2 0,2 0,1 R R R R 0,1 0,1 0,2 0,2 0,1 0,1 0,1 0,1

Zero mean white gaussian

(variance)

Code

version

Speckle

(variance)Poisson

Salt & Pepper

(noise density)

Fig. 18 Speckle noise in QR codes for 250 characters for L error correction level and M error correction with variance 0.04, 0.2 and 0.3

23

Another thing observed in this experiment was the “deterioration” of the message when being read. In

higher versions the message became more and more deteriorated even if it was still readable in most

cases, often times, characters missing or being replaced by other characters.

Fig. 19 Shot noise (Poisson) in QR codes for 250 characters for L and M error correction levels

Fig. 10 Salt and Pepper noise for 250 characters for an L error correction level with noise density of 0.1, 0.2 and 0.7 respectively

Fig. 21 Zero Mean White Gaussian noise with 0.01 variance for 250 and an L and M error correction levels for 250 characters

24

3.2 Multilevel quick response codes

As stated before, one of methods used to increase the capacity of QR codes was to use multilevel

spectral multiplexing. In this method, different approaches are taken to creating the QR codes such as

choosing different intensities for dark and light modules in the YCbCr color space or choosing module

color based on the its bits using the RGB color space.

Using the first approach, it uses the bases of RGB color by having four levels of intensities which allows

64 different colors to be used, but since the decoder cannot distinguish all of these it is necessary to

choose a set of colors from the available pool [12]. By doing this it was reported that the capacity of the

code increased four times by using 16 colors.

Fig. 22 Multilevel intensities for a grayscale QR code [10]

Fig. 23 Multilevel intensities for color channels in the conventional QR code [10]

The approach in itself is very similar to the approach used for multiplexing three different codes into a

colored one. First two intensity levels for the dark and light modules in each color channel are chosen

for the YCbCr color space in which each value should have a maximum distance between each other

so they can be distinguished from each other, and afterwards they are converted into the RGB color

space.

25

{

𝑅 = 1.164(𝑌 − 16) + 1.596(𝐶𝑟 − 128)

𝐺 = 1.164(𝑌 − 16) − 0.391(𝐶𝑏 − 128) − 0.813(𝐶𝑟 − 128)

𝐵 = 1.164(𝑌 − 16) + 2.018(𝐶𝑏 − 128)

Fig. 24 Example of a colored QR code using the method described [10]

When decoding, the RGB color space is converted back to YCbCr color space, separating the color

channels into Y, Cb and Cr and then decoding the bicolor QR code in each color channel [12].

{𝑌 = 0.257𝑅 + 0.504𝐺 + 0.098𝐵 + 16(𝑌 − 16) + 1.596(𝐶𝑟 − 128)

𝐶𝑏 = −0.148𝑅 − 0.291𝐺 + 0.439𝐵 + 128𝐶𝑟 = 0.439𝑅 − 0.368𝐺 − 0.071𝐵 + 128

This approach is compatible with current decoders, because when decoding, only the luminance of the

images are used and the chrominance is ignored.

The second approach allows for a standard to be set for multilevel spectral multiplexing. By choosing

the intensity level based on the value of the bit associated with that module and the numbers of QR

codes instead of a random number chosen by each encoder, it will lead to far less errors that might

occur from the first approach and is considered as the best out of two to perform the simulations on.

26

The first part of the simulation was to try and multiplex 2 or more codes into a single color QR code. In

order to do this we divided the red part of RGB into 2 spaces for multiplexing 2 codes. This process

begins by loading each of the images into Matlab using the imread function, giving us a XxYx3 matrix

where the 3 corresponds to the RGB color spaces where 0 is black and 255 is white. After this is done

we chose either the red, green or blue color space as the color we will be working with and transform

the matrix that corresponds to it all to 255. This will make it so that if the color we are working on will be

red (255 0 0), blue (0 0 255) or green (0 255 0) where the black modules were. Afterwards we transform

each of the images to a fraction of their color space in order for them to be added to each other

afterwards. In this case if only the first code was active it would stand at 2/3 of the color, if only the

second one was active it would stand at 1/3, making so that if both were active it would be 0 and if

neither was it would be 1, leading to the images below.

Fig. 25 Examples for colored QR codes using a single color for all the codes

From these images one would say that choosing grey or blue would be the best choice since it’s easier

to ascertain each of the codes, while for green and red some of the lighter areas might be misinterpreted

as white, thus decoding the message with error, but from readers it has been proven that they can in

fact receive part of the correct message.

Now regarding multiplexing more than 2 codes for the same color space we would need to divide the

space in such a way that their sum would lead to 1 but without overlapping colors. One of the ways to

27

accomplish this is by having one of the codes above half of the color space, and the other codes below

it with distinct values as well, so that their total sum would be 1.

{

𝑥 + 𝑦 + 𝑧 = 10 < 𝑦 < 0,50 < 𝑧 < 0,5𝑥 > 0,5𝑦 ≠ 𝑧

The problem with this method is that the 2 codes that stand in the less than 0,5 part of the color space

might not be distinguishable between themselves. If let’s say one of the codes is at 0.1 it might be

mistaken as a white module and thus not be accounted for, thus creating a false message.

Another way to go on about this method is by using bits for the amount of codes that we wish to encode.

This is done by first assigning a space in the RGB color space to each code, by using the formula,

2𝑖(2𝑛 − 1)⁄ , where “n” is the number of codes we have and “i” is a value from 0 to n-1 where 0

corresponds to the first code, 1 the second code and so on and so forth. This method allows for any

number of codes we desire but with each increase, the codes that have a more “light” color start to

become hard to be distinguished among themselves. If the decoder receives the message in its data

form, this won’t pose a problem, but if a picture must be taken first, depending on the quality of the

camera, the message will not be correctly decoded.

Table 8 QR code values for amount of codes being multiplexed

Values for each code being multiplexed

Number of codes being multiplexed

1st code 2nd code 3rd code 4th code 5th code 6th code

1 1 0 0 0 0 0

2 1/3 2/3 0 0 0 0

3 1/7 2/7 4/7 0 0 0

4 1/15 2/15 4/15 8/15 0 0

5 1/31 2/31 4/31 8/31 16/31 0

6 1/63 2/63 4/63 8/63 16/63 32/63

28

This time, 4 QR codes will be multiplexed. The Matlab process for this is very identical to the one used

before with some minor adjustments. First, each of the QR codes is converted to a selected color space,

in this case, the color space was the blue one.

Now that each of the QR codes in in the blue color space, they will be multiplied by a value listed in

Table 8, that is, 1/15 for the first code, 2/15 for the second one, 4/15 for the third and 8/15 for the forth.

The lower the value of the QR code, the “darker” it will be as seen in Fig.27, since the closer the values

are to 0 the closer they will be to the color black.

The QR codes can now be “added” by performing the imadd command that will add each of the images

to form the final QR code as seen in Fig. 28 on the right

Fig. 26 QR codes in blue color space

Fig. 27 QR codes in blue color space after being multiplied by their respective values

Fig. 28 "Adding” of previous QR codes

29

Regarding the results of this simulation, only the first QR code was “read” by the decoders due to the

way that they are programmed to recognize the darker modules as “black” and the rest as “white.

For smaller QR versions, like versions 1 to 3 it may be advisable to just use higher QR code versions to

transfer data if they are available since the increase in capacity is still surpassed by the higher versions

even after the multiplexing, not to mention the additional time it may take to encode and decode the QR

codes multiplexed.

3.3 Multiplexing with color coding

Another possibility to perform the spectral multiplexing is by using color coding. The way this multiplexing

works is by verifying the values of pixels in the QR code, since there will be a distinct color for each

pattern possible. An example of possible patterns and the respective colors for 3 QR codes is presented

in table 9.

Table 9 Possible color combinations for a color QR code

QR code 1 QR code 2 QR code 3 Index Red Green Blue Color

0 0 0 0 0 0 0 Black

0 0 1 1 0 0 1 Blue

0 1 0 2 0 1 0 Green

0 1 1 3 1 0 0 Red

1 0 0 4 0 1 1 Cyan

1 0 1 5 1 0 1 Magenta

1 1 0 6 1 1 0 Yellow

1 1 1 7 1 1 1 White

30

Fig. 29 Multiplexing and color coding table for 3 QR codes [3]

If the number of QR codes being multiplexed increases, the color coding table will change and more

distinct colors will be required, such as 16384 colors for 14 QR codes [3]. To generate the color coding

tables becomes then a time consuming and impractical task, but it can be tackled by using MATLAB

tools.

Experiments conducted on this method verified that the data capacity can be increased up to 24 times,

thanks to the 16,777,216 colors available from the RGB color space for 8-bit representation ( 2^24), but

multiplexing over 10 images is not advised since the time to multiplex and demultiplex increases with

each QR code added for multiplexing[3].

Some studies have also considered the usage of symbols instead of colors in the multiplexing and it

follows the same principal of constructing a coding table for each of the possible patterns, but there still

aren’t any major breakthroughs using this method compared to using color so it’s slightly disregarded in

research [13].

31

For the simulation, it was attempted to recreate the multi colored QR code presented. At first a simpler

colored QR code was produced by using only 2 codes for multiplexing. This was done by each of these

codes having a distinct color space and the third space being all white leading to the results in Fig.30.

This simulation could be done as well for any other color chosen as the third color space and isn’t

restricted to white as in the results above.

Fig. 31 QR color code using the first code for red and blue and second code for green

For 3 QR codes, much like in the first simulation, these are loaded unto Matlab but this time, each one

is assigned to a different color, in this case the first code would be red, the second green and the third

blue. The different colored QR codes are then multiplexed like before to create the multi-colored code

in Fig. 32.

Fig. 30 Examples of multiplexing 2 QR codes for different color spaces

32

Fig. 32 Multiplexing of 3 QR codes into one where each code has a different color space

This code, just like in the multilevel multiplexing, has more information in it than a singular QR code of

the same version, and is easier to ascertain where each of the 3 codes are, even when using a camera,

but the current QR code readers do not have the ability to do so and are unable to even form part of the

message like it did when it was used for multilevel where at least the first code was read.

The problem for this method would also be that in order to multiplex more than three codes, there would

have to be a particular type of color for the extra codes and these could not be the combination of any

of the 3 previous codes, nor could they originate another color that already exists and is being used

when added to another code like purple or yellow.

33

3.4 Multilevel color spectral multiplexing

A simple solution to the problem for the color coding would then be to merge the 2 methods presented

in this section, that is, to use the same color but “lighter” for the new codes, thus while the color

technically would be the same, the different “aspect” of the color would allow to circumvent the problem

and allow for multiplexing of more than just 3 codes.

The way this is implemented in Matlab is to use the bit equation posted above for each pair of QR codes

(1-2, 3-4, 5-6 for example in the case of multiplexing 6 QR codes) where each pair has a different color

assigned to them and each of the codes in that pair has a different tone of that color assigned by the bit

equation. Afterwards the codes are multiplexed together just like in the previous section originating the

QR code seen below.

From the figure above it is possible to observe that this last method is probably the best out of all the

listed, since each of the codes are clearly distinguishable even if they have a “lighter” color than some

of the other ones, unlike if only one color scheme was used, also unlike the color QR code method there

Fig. 33 Example of multiplexing 6, 9 and 12 different codes into one

34

is no need to specify any other color besides red, green and blue since all of the codes will be one these

three and the multilevel part of the encoding will make sure that when multiplexed, no color will be

repeated.

It is of note as well that while in this simulation the QR codes were paired in a simple way so that it was

easier to decode, there is no need for this to happen in further development of this method, since if the

person encoding so chooses, they can put all the codes except one in the same color space and this

method would still work.

This method only requires that either a standard is set in further development for the order of the

decoding to occur, and for a means to transmit the information of how many QR codes are in that color

space to hasten the decoding, although this last part can be skipped at the cost of spending a couple of

seconds analyzing the values in the matrix and the program figuring out for itself based on them, how

many QR codes are on the color space.

For the final part of this simulation, a Matlab program was created in order to de-multiplex and decode

the final message transmitted in this type of code, which performed the following tasks in order to decode

it:

1. Separate the codes from the original multiplexed one by separating the color spaces and

observing how many different values exist and performing log2 𝑛 where n is the total of different

values;

2. Obtain the EC level, mask pattern and version of the code;

3. Substitute values that won’t be important to the decoding of the message so it won’t interfere

with the decoding, such as the finder, alignment and timing patterns and the dark module and

seperators;

4. Remove the masking pattern applied on the code;

5. From the number of total bits, perform a for cycle that will read the code and save the bits in an

array;

6. Convert array from bits to characters to obtain decoded message.

Since each of the bits had a specific value assigned to them, and the image was received from one

previously created and not acquired through a camera, the message did not require the error correction

bits to be considered since no damage had occurred and was decoded correctly.

4 Large capacity QR Code

35

4.1 Image transfer

Now that it was shown that a larger message can indeed be encoded in QR codes through multiplexing,

let’s take a closer look at the data capacity and the time it takes to multiplex and demultiplex these

codes.

In a normal QR code, about half of its bits are error correction bits so that in case the QR code gets

damaged, the message can still be received. Although the error correction bits are obviously important,

it is also space that could be used to include more information, if the QR code would not be damaged

in any way.

Let’s consider then that the QR code would be transferred in a digital way that would not get damaged

then: each code has a RGB part to it, this means that their theoretical total capacity would not be,

((𝑣 − 1) ∗ 4 + 21) 2 ,where v is the version of the code, but instead three times that, so while a version

1 QR code has 441 bits if purely seen by the first equation, it actually has the ability to hold 1323 bits if

we make use of the three RGB color spaces.

But although this is the highest amount of bits a version 1 QR code could hold, it still needs its finder

patterns, alignment patterns, separators and timing patterns in order to be a recognizable QR code,

which will reduce the amount of bits by around 606 bits of the 1323 estimated total in this case.

Considering that the total amount of bits for a byte mode in a version 1 code is 136, by using the RGB

color spaces and disregarding the error correction bits, the amount of data for this version would

increase by 5.5 times.

Of course this is only considering that only the color multiplexing could be used. As it was proven,

multilevel also works to increase the capacity of the QR code while working in conjunction with color

multiplexing, so the capacity of the QR code could become far greater than just by using color

multiplexing.

The only limitation to this is the time that it takes to perform the multiplexing and demultiplexing of the

QR codes. As was stated in the section 3.3, by multiplexing and demultiplexing 10 QR codes and and

performing the decoding, it took over 20 mins for the task to be completed, which is too much time to

decode a QR code that is known for its quick response, but once again this is considering that the error

correction and masking occurs for each of the codes so that the message is decoded correctly.

If the QR codes can actually hold such high amount of bits, they could potentially hold and transfer the

data of other things such as images or even sound, if their data can be converted to bits or are already

in bit form.

A simulation was performed for both these cases, to see how much time it would take to encode/decode

each and if it was possible to fully recreate the original piece.

36

In the early stages of the implementation, a 50x50 JPEG was chosen since its size would “fit” into several

of the large QR codes simply by putting the RGB values inside, without the need for them to be converted

into binary.

For the first step in this stage, the smallest version QR code possible was found on the basis that the

total number of bits hard to be larger than the width times the height of the image being inserted, which

in this case was 2500 bits long at least.

Since at this stage the focus was more on the amount of data transferred, the alignment patterns were

ignored and only the location and timing patterns were included.

The image created included a bit representation at the start that would identify the height and width of

the original image in order for it to be recreated, followed by the RGB values of the image being tested

as shown in fig.34.

The image was in fact decoded correctly using this method of inserting its RGB values into the QR code,

but the problem arose for bigger types of images such as 403 by 530, that did not fit inside of a version

40 QR code.

Multilevel was then considered as a way to tackle this problem since it was proven to be an efficient way

to increase the capacity a QR code, but it carried a new problem with it, since that while before when

using multilevel the only values multiplied were 0 and 255 to be used in the multilevel QR codes, now

there were all the ones in-between those two values and there was not an easy way or even fail proof

way to recover them once multiplied since the program would narrow each value after the multiplication

had taken place, so a new approach had to be taken.

The best approach was then to use the same way normal message were encoded and that was by

transforming the data into bits. Since all the values in the image go from 0 to 255, an 8 bit string for each

of these values was enough, but that obviously meant that the data to be inserted into the QR had to be

increased by 8 times

Fig. 34 50x50 image and QR code holding its information

37

A version 18 QR code was then chosen to be multiplexed by 9 different QR codes, 3 for each of the

color spaces since it could hold a total of 23763 bits for the images’ required 20000 bits, but a smaller

code can be chosen if need be, by increasing the amount of QR codes to be multiplexed.

The alignment patterns were also included to be able to distinguish each version of QR codes since it

would also help with possible image distortions.

After the size of the data and the reserved areas were placed, multilevel was applied to each of the RGB

color spaces, and the QR codes multiplexed together into fig. 35.

For the decoding, like for the multilevel method, the reserved areas are altered in order for the cycle to

ignore them when scanning the QR code. After receiving the dimensions for the image, the code is

scanned X amount of times until the total amount of bits received by the images’ dimensions is reached.

With each iteration of the scanning, it considers a different QR code is ‘active’, as in for the first iteration,

the first QR code is considered ‘active’, and searches for the values in the matrix that corresponds to

the possible combinations for that QR code for bit 0 and bit 1 as seen in table 10.

Table 10 RGB values for 3 QR codes being multiplexed

RGB Values Bits

Fig. 35 Version 18 QR code holding image information

38

1st code 2nd code 3rd code

255 0 0 0

108 0 0 1

180 0 1 0

36 0 1 1

216 1 0 0

72 1 0 1

144 1 1 0

0 1 1 1

As seen in fig. 35, there is a “lighter” part of the code on the left. This is due to the third QR code not

being fully filled with data since it was not necessary, and so the left over bits were all considered as 0.

The encoding ended up taking 10 seconds to multiplex all 9 different QR codes and less than 3 seconds

to demultiplex and reconstruct the original image.

From this simulation it is possible to conclude that indeed this type of data can be transferred in QR

codes using the method above although for this image, the bytes were less than the bytes of the image

produced. This can possibly once again be circumvented by using a lesser version QR code and

increasing the amount of QR codes multiplexed, thus reducing the byte size of the image at the cost of

a slightly higher encoding and decoding time.

For the 403x530 JPEG previously mentioned it was noted that it required over 162 QR codes being

multiplexed for a version 40 QR code (54 for each RGB color space). This would lead to an enormous

time for encoding and especially decoding the QR code to recreate the image.

If the aim of the person encoding is simply to transfer the data without worrying for the

encoding/decoding times, this method can be used for an image of that size, since the byte size of the

QR code will be less than that of the image being encoded.

39

4.2 Audio transfer

Regarding the sound file simulation, a MP3 file with the duration of 1.5 seconds and 23 Kb was chosen

for this simulation.

Like in the image simulation, the first step is to figure out how much data is going to be transferred. This

was performed with Matlabs’ audioread function which gave us 131318 entries (65659 by 2), since the

audio file was in stereo. For the simulation only the first column was considered for the encoding.

The task now was to figure out how to transfer said audio data into the QR code. Unlike the image, the

audio file didn’t possess values that would translate easily into bit form, since they were comprised from

-1 to 1 and some of its values went as low as 10−6.

The way this was tackled was by converting these values into more manageable ones. For negative

values, they were multiplied by -150, thus allowing them to be converted into bit form and making it so

that lower values could be “seen” as well. For the positive numbers, they were multiplied by 150 and

added 150 afterwards.

This allowed for the data to be encoded as 8 bits, but for the lower values this was still not enough to be

able to be distinguished and so they were considered simply as 0 for the remainder of the simulation.

The 40 QR code version was chosen, since it is the largest QR code available, and while it is possible

to encode the data with smaller QR codes, this version needs 18 different codes to be multiplexed in

order for the whole data to be transferred, which means that for lower versions, like 30, 20 and 10, the

amount of codes necessary would increase by 1.7, 3.4 and 10.8 times respectively.

The creation of the QR code is basically the same as the one described for the image coding. We begin

by first reserving the areas for the several patterns to be inserted into each of the 18 codes and then

start to insert the audio data itself.

The first information regarding the data inserted is its total size, just like it was for the image, so that

the original audio can be recreated. After this is done, the data for each entry can be inserted in its bit

form, performing several ‘for’ cycles to insert the data into the 18 different QR codes.

Unlike the image simulation, the audio does not have an RGB color space so there is no need to

specifically insert the bits in different color spaces like in the image simulation where it helped the

decoder to have it be like that.

For the audio file, the data can be inserted in anyway without needing to be ‘restricted’ to the color

spaces, meaning that if the person encoding wanted to use all 18 codes in the red space, they could

do so, but for an easier way it was considered that it was better to divide the codes through all 3 color

spaces, since 18 is divisible by 3. If the number of codes was not divisible by 3, then one or two color

spaces would have extra QR codes assigned to it.

40

After the data was inserted into the QR codes, the patterns were added and each pair of 6 QR codes in

each color space is converted so that they can be multiplexed like in the image simulation.

For the decoding of the QR code, first each color space is separated, the amount of data is acquired

and the pattern areas are altered to a value that won’t interfere with the reading of the QR code.

For the retrieval of the data inserted in all 18 QR codes, 6 ‘for’ cycles are performed for each of the color

spaces to retrieve the bit values for the QR code associated with that cycle, as seen in the image

simulations’ table 10.

The initial results yielded a lot of noise due to problems with smaller values having the same ones as

the negative ones, such as 0,00005 ∗ 150 + 150 = −1 ∗ (−150), since in Matlab the values are rounded

up. The way this was solved was by considering that values with less than 2 decimal points were ignored,

thus reducing the noise that occurred.

Another problem that arose in the decoding was the fact that a column was repeated in column 92,

which lead to an initial error while decoding. There is no easy way to deal with this since it may be

possible that the repeated column may not be a repetition and in fact a correct column. The column was

possible to be identified since it was next to an alignment pattern and it made the alignment pattern into

a 5x6 instead of a 5x5.

The encoding for this audio file took a minute to complete while the decoding took almost 2 minutes to

demultiplex it and reconstruct the audio file.

Fig. 36 Version 40 QR code holding data file information

41

Although it is possible to distinguish the differences between both versions, the decoded version was

audible and presented no noise even if the values weren’t identical to the original.

0 0.5 1 1.5-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time (s)

Am

plit

ude (

au)

Fig. 37 Audio file and reconstructed audio file

0 0.5 1 1.5-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time (s)

Am

plit

ude (

au)

42

The result was also put under SNR and MSE analysis, for the original and the reconstructed audio file,

by applying their respective formulas bellow, where x(n) is the original audio file and y(h) the

reconstructed audio file. The estimated values for SNR is 36,03 dB and 0.0000134 for the MSE.

𝑆𝑁𝑅 = 10 log10(∑ 𝑥(𝑛)2𝑛

∑ (𝑥(𝑛) − 𝑦(𝑛))2𝑛

)

𝑀𝑆𝐸 = ∑ (𝑥(𝑛) − 𝑦(𝑛))2𝑛

𝑛

The ability to ‘insert’ audio files may be a useful way for advertisement companies or the music

companies to distribute small audio files like ring tones, without the consumer needing to download the

file from the internet.

43

5. Conclusions

From the results obtained in chapter 3 of the thesis, it is possible to verify that using spectral multiplexing

on QR codes is a viable method to increase their capacity by at least 24 times. By using this method, a

smaller version code can transmit the same amount of data that would take to be transferred in a higher

version code, while maintaining the amount of bytes that the smaller version has. This would allow for a

possibly more economic approach for companies to divulge their products or for hospitals to add more

information about their patients to their bracelets.

The possibility of audio being transmitted in a QR code by using spectral multiplexing could open up a

new venue for the music industry and the advertisement industry to divulge their works by creating QR

codes that have audio files in them, such as ringtones, so that when someone scans the code, that

person could decode the QR code and listen to the audio file with almost no loss in quality, without

requiring an internet connection.

Since there already exists some types of colored 2D codes, the code used for the readers could maybe

be applied to the ones that decode QR codes, else the way one would go about changing the way the

code works for the QR code readers, would be to separate the image into its RGB color space, verify

how many QR codes were multiplexed by using the formula, 𝑡 = 2𝑛 , where t is the different values

presented in the color space and n is the number of QR codes that were multiplexed, and separate

these values into each of their original QR codes and then read them as normally.

The only limitations this method has are the printing of QR codes since different devices from different

manufacturers reproduce the RGB color space in different ways, the dependency on camera phones

for capturing the image in the most adequate illumination and brightness and the amount of time that it

may take to decode a multiplexed QR code with a high amount of QR codes in it, but this can be

circumvented by some form of code optimization and a faster processor, along with if the means of the

data transmission is purely digital, the camera and printing problems can be disregarded.

44

45

References

1. Max E. Vizcarra Melgar, Alexandre Zaghetto, Bruno Macchiavello, Anderson C. A. Nascimento,

“CQR CODES: COLORED QUICK-RESPONSE CODES”, 2012 IEEE Second International

Conference on Consumer Electronics - Berlin (ICCE-Berlin), 2012

2. Hiroko Kato,Keng T. Tan,Douglas Chai, “Barcodes for Mobile Devices”, Cambridge University

Press, 08/04/2010

3. Hiren J. Galiyawala and Kinjal H. Pandya, “To Increase Data Capacity of QR Code Using

Multiplexing with Color Coding: An example of Embedding Speech Signal in QR Code”, 2014

Annual IEEE India Conference (INDICON), 2014

4. Denso ADC, “QR code Essentials”, 2011

5. Jose Joaquin Mira, Mercedes Guilabert, Irene Carrillo, Cesar Fernandez, Ma Asuncion Vicente,

Domingo Orozco-Beltran, Vicente F. Gil-Guillen, “Use of QR and EAN-13 codes by older

patients taking multiple medications for a safer use of medication”, INTERNATIONAL

JOURNAL OF MEDICAL INFORMATICS, 2015

6. Yanghua Gao, Zhihua Zhang, Huanwen Wang, Hailiang LU, “Two-Dimensional Code Based

Spatiotemporal Modeling and Its Application in Object Tracing”, INTERNATIONAL JOURNAL

OF DISTRIBUTED SENSOR NETWORKS, 2015

7. Artur Carnicer, Amir Hassanfiroozi, Pedro Latorre-Carmona, Yi-pai Huang, Bahram Javidi,

“Security authentication using phase-encoded nanoparticle structures and polarized light”,

OPTICS LETTERS, JAN 15 2015

8. Hou A-Lin, Feng Yuan, Geng Ying, “QR code image detection using run-length coding”, 2011

International Conference on Computer Science and Network Technology, 2011

9. Dr. Philippe Cattin, "Image Restoration: Introduction to Signal and Image Processing". MIAC,

University of Basel, 2012

10. Danny Pascale, “A Review of RGB Color Spaces … from xyY to R’G’B’”, The BabelColor

Company, 2002-2003

11. Sabine Süsstrunk, Robert Buckley, Steve Swen, “Standard RGB Color Spaces”

12. Masanori KIKUCHI, Masaaki FUJIYOSHI, and Hitoshi KIYA , “A New Color QR Code Forward

Compatible with the Standard QR Code Decoder, Intelligent Signal Processing and

Communications Systems (ISPACS), 2013

46

13. Sartid Vongpradhip, “Use Multiplexing to Increase Information in QR Code”, The 8th

International Conference on Computer Science & Education (ICCSE 2013), April 26-28, 2013