How to increase reach efficiency and effectiveness of MEO’s digital marketing campaigns?

(1)

A Work Project, presented as part of the requirements for the Award of a Master Degree in Management from the NOVA School of Business and Economics.

Field Lab MEO – How to increase reach efficiency and effectiveness of MEO’s digital marketing campaigns?

Measuring the performance of MEO’s digital marketing display campaigns

MIGUEL PINTO TINOCO FRAGA

STUDENT NUMBER: 2098

A Project carried out on the Master in Management Program, under the supervision of:

Prof. Elizabete Cardoso

(2)

2

Abstract

This thesis report follows a Work Project (WP) developed at the leading Portuguese telco (MEO) on the topic of performance measurement of digital marketing (DM) display campaigns. The WP ought to solve issues such as lack of coordination, internal misinformation, data fragmentation and lack of return-centricity of MEO’s digital campaigns’ department. In the end, with a key focus on the use of Web Analytics, a concrete process for campaign ROI calculation was developed, as it was an identified gap, along with several complementary recommendations aiming at the optimization of the firm’s digital performance measurement capabilities.

Keywords: Digital marketing, Display advertising, Performance measurement, ROI

1. Introduction

This Master thesis aims at discussing and analyzing the process and the outcomes which resulted from a Field Lab (FL) project carried out for approximately 12 weeks at the main headquarters of MEO – Serviços de Comunicações e Multimédia, S.A., a major powerhouse in the Portuguese telecommunications market. The project resulted from a partnership between Nova School of Business and Economics (Nova SBE, “the school”) and MEO (“the company” or “the client”), with the goal of having Master thesis students identify potential for growth and optimization in digital marketing. The need for this collaboration, on the company’s side, arose due to several factors. The most notable are the series of deep internal changes that MEO underwent a few months prior to the launch of the project, the growingly intense competition on the Portuguese telecom spectrum and the evolving nature and complexity of marketing activities in general. This way, the WP aimed at developing numerous paths in which MEO could improve and suppress eventual gaps on the DM field, mostly regarding the efficiency and effectiveness of its digital marketing presence, with a major focus on its display campaigns.

The following sections of this document – context, literature review, methodology, discussion of findings, recommendations and future research and limitations – attempt to give a clear idea of how the main principles and frameworks applied led to the conclusions that were reached. The three last sections, as well as the literature review of this document, will be mostly focused on one of the key individual topic of this WP: the measurement and evaluation of MEO’s digital marketing display campaigns. Practical implications of the work developed throughout the project’s duration will be analyzed and discussed, as well as the theoretical basis behind those implications.

2. Context for the project

In order to further contextualize this Work Project’s (WP) topic, effective usefulness and real-life application, it is necessary to approach the issue according to two equally relevant sets of happenings and facts: the company’s general complex situation since June 2014, which led to a

(4)

4 turmoil with no precedents in its history, and the still incomplete and gapped process of evolution of the digital marketing area of activity within the company.

2.1. The company’s context

2.1.1. MEO integrated in the history of Portugal Telecom

Although MEO is the major player in the Portuguese telecom market, the acronym that gives the company its current name did not exist until 2007: After a hostile takeover proposal by Sonaecom to Portugal Telecom (PT), a spin-off separated PT Comunicações (PTC) from PT Multimédia (PTM) led to PTM turning into Zon Multimédia and becoming a key competitor on the Cable TV area. PT lost its position of unquestioned leadership on that market, opting for creating a new brand to rival with Zon. MEO was born as an innovative brand, integrated on the PTC business.

MEO’s still short history is, therefore, inseparable from the historical path of Grupo PT, its holding. PT is the most significant reference of the Portuguese telecom scene since its appearance in 1994, having been born out of the merger of three state-owned telecom companies. Two years later, the government began its privatization process, which underwent five stages of capital alienation until 2000. It was finished in 2011, with the demand from the Troika that financially intervened in Portugal at that time that the state would lose the ‘golden share’ it held on PT. In the meanwhile, PT’s history is marked by some feats and achievements that make it a Portuguese reference, mostly related to its level of R&D investment, whether in terms of technologies, products and services offered or the establishment of partnerships with important international players, aiming to expand its reach and scale. Relevant examples are: a) its national historical implantation in all major products and services, mostly television, internet, mobile and fixed communications; b) the joint-venture formed in 2003 with Telefonica (major Spanish “telco”) to co-found Vivo, a from-then-on leading operator in Brazil (the partnership lasted until 2010, when Telefonica bought PT out); c) the merger between PT and Oi, a major telecom company from Brazil, signed in 2013 with the goal of creating a huge Portuguese-Brazilian company to operate in every Portuguese-speaking country.

(5)

5 2.1.2. The acquisition of Portugal Telecom in 2015

The deal celebrated with Oi, however, did not generate the initially desired effects, and difficulties in reaching a definitive agreement – aggravated by a scandal that emerged meanwhile and saw PT make an unsuccessful investment in commercial paper of Rioforte, one of Grupo Espírito Santo’s holdings – led to the celebration of a deal which allowed for Oi to sell the controlling shares of PT which the Brazilian company had acquired. In 2015, the Altice group (French) acquired 100% of the company’s shares, leading the way for a still ongoing restructuring phase. Naturally, this restructuring phase has led PT to face numerous challenges, with impact on all areas of activity, from which Digital Marketing is no exception. This was a reason behind the need for the team’s contribution and help – in developing recommendations and setting fresh perspectives for the future, amongst a period of higher uncertainty – throughout the three months which it lasted. 2.1.3. MEO’s current offering and market position

MEO soon started attempting to take the market by storm using high-profile brand ambassadors (humorist group Gato Fedorento) and high-impact campaigns. It gathered three functionalities in one package: cable TV, telephone and internet access (with optical fiber since 2009). In 2013, MEO innovated by offering a quadruple service which added mobile phone service, extending to a 5P service in July 2014 (with the addition of mobile internet access). In the beginning of that year, MEO replaced TMN – PT’s historical mobile phone brand – and in 2015 MEO was merged with PTC, resulting in a single company named MEO – Serviços de Comunicação e Multimédia, S.A., responsible for managing all of PT’s commercial offering. Currently, there is only one other strong B2C brand within the company: MOCHE, at the mobile communications level, targeting the under-25 segment of the population, with a bolder message. The focus of this WP, though, is merely the MEO brand and its display campaigns, a delimitation which was made from the start.

As a complement to its main offering, MEO has also developed a range of complementary products and services as an attempt to improve users’ experiences while watching TV, surfing the web or using a smartphone (e.g.: MEO Drive, MEO Go, MEO Music and MEO VideoClube). These are

(6)

6 seen as a way of distinguishing the three main telecom brands – MEO, NOS and Vodafone – from one another, since their main offering has various similarities, in terms of contents and pricing. In terms of market presence, MEO’s growth has been remarkable. As of June 2015, MEO was the leader of the mobile communications market, with a 47,5% share (Alves, 2015), as well as of the mobile and fixed internet one, with 48,6% and 47,7% market shares, respectively (Ribeiro, 2015). Simultaneously, it has gotten close to NOS on the cable TV market, with a 41,4% share vs. the 43,7% of the market leader (also June 2015 data, from ANACOM (2015)).

2.2. MEO’s digital marketing roadmap

Complementary to its top management changes, MEO is in the midst of a “digital transformation” with impact on the management, analysis, reporting and strategizing of its digital marketing campaigns, a process which is described internally as a “5-steps data-driven approach”: 1. Adopt a unified technology platform (campaign management); 2. Attack fragmentation (management, analysis and reporting); 3. Implement advanced techniques (management); 4. “Bring the math men to the table” (analysis and reporting); 5. Test and learn (analysis, reporting and strategizing) From these, none had yet been implemented with total success and completeness, although the three first ones had already been started. This allows to further understand why MEO’s digital marketing team felt that a WP such as this one could be of benefit and value for both the group of students and the company, as there were still numerous issues to address and recommendations to be made regarding MEO’s presence on this field. This individual report’s major focus is, as mentioned before, on the topic of measuring the performance of MEO’s digital marketing display campaigns, thus aiming to suppress gaps from steps 2 and 4 from the list above. The following exercise of literature review will aim at describing and analyzing findings and approaches to that issue in order to provide scientific and solid context to the “Findings” and “Methodology” sections.

3. Literature review

Digital marketing: One aggregating definition of digital marketing is referring to it as “The use of digital technologies to create an integrated, targeted and measurable communication which helps

(7)

7 to acquire and retain customers while building deeper relationships with them” (Smith, 2007, in Royle & Laing, 2014). Its tremendous dynamism and rate of evolution (Ryan & Jones, 2011) have led to it being a growing subject of study, making it natural for new attempts of conceptualizing this issue to appear every now and then. The focus placed on relationship building with customers as a key objective is not exclusive of digital marketing, though. Actually, customer relationship management (CRM) is seen as an absolutely fundamental concept of marketing in general, dealing “with all aspects of acquiring, keeping and growing customers” (Kotler & Armstrong, 2011). Digital marketing in the telecommunications sector: In fact, when reflecting on the steps ahead for the telecom industry due to the rise of the digital era, Banfi, Caylar, Duncan, & Kajii (2013) identify a fundamental challenge that alludes precisely to those three aspects. It is argued that such challenge “involves creating brand engagement through digital media and platforms, turning that brand engagement into brand preference, and leveraging it to drive sales and loyalty”, following the increasing importance of digital touch points in the contemporary Customer Decision Journey (Appendix 1), when compared to traditional ones (Banfi et al., 2013). This WP involved direct collaboration with a telco’s digital campaigns’ department which has identified precisely that as simultaneously a key challenge and a main goal of its performance. Theorists in this field, however, point out to the fact that the main problem telecom companies face is a broader one, at the upstream level: “the lack of an integrated strategic approach to DM” (Royle & Laing, 2014), which has been a prevailing theory in recent years, showing no significant developments. Effectively, some years before, authors such as Ryan & Jones (2009) had also covered the need for a DM strategy to be straightly directed towards a business’ strategic goals. Valos, Ewing, & Powell (2010) have corroborated the existence of this problem and attributed it to three key issues: “diversity in current and emerging online applications (…), measurement issues, and skills shortages”.

Performance measurement in digital marketing: The portion of the WP to which this thesis report is dedicated concerns precisely one of the three aforementioned issues: the measurement of the

(8)

8 outcomes of MEO’s digital marketing efforts. In recent years, there has been a spurt of work published on the topic, as a consequence of the recognition of its growing importance by academia and business players. Empirical proof of a positive impact of marketing performance measurement on a firm’s performance has been developed (O’Sullivan, Abela, & Hutchinson, 2009), although most of the relevant existent literature on the topic – which ranges from the turning of the 21st century to more recent publications and advocates the absolute need for measurement of DM activities (Spencer & Giles, 2001) (Epstein & Yuthas, 2007) (Tonkin, Whitmore, & Cutroni, 2010) – is based on qualitative premises and business experience. It is now clear and consensual among writers that campaign performance evaluation is a key driver for success in the new digital age, but questions about how to properly implement it remain for MEO. Mayar & Ramsey (2011) describe the problem that firms face as one of “lack of adequate metrics and measurement systems to drive marketing performance”, which is applicable for this specific case. For MEO, the greatest identified challenge was measuring the performance of campaigns which have a higher focus on building brand awareness rather than on generating E-Commerce conversions per se, which is also a topic of growing research interest. Epstein & Yuthas (2007) acknowledge that, in these cases, “outcomes of IM [Internet Marketing] are more complex” and that the “understanding of both the short and long-term payoffs associated with IM investments can benefit organizations enormously”.

As higher percentages of the total advertising budget worldwide are growingly dedicated to digital advertising (forecasted to grow 15,7% by the end of 2015, which will make up for a 24,3% share of the US$529 billion global advertising budget, according to Carat (2015)), the interest companies place on optimizing their DM measurement systems also grows. While this may suggest that the supply of consistent theoretical and empirical results could increase in accordance, it is difficult to find two publications on this issue placing focus on and presenting the same ideas from start-to-finish of the measurement process, making it difficult for consensual patterns to emerge. Spencer & Giles (2001) have focused on the distinction between impact measures and influence measures,

(9)

9 according to a company’s objective when launching a DM initiative (the first ones when the objective is “generating visitors to the site” and the others when “the aim is to build relationships with a defined audience”). Epstein & Yuthas (2007), on the other hand, cover the topic by attempting to build “a process for evaluating the financial performance of Internet marketing” and describing prior approaches to performance measurement which they consider to have failed: the ‘clicks-and-hits approach’, according to which “the organization tracks IM-related user behavior” and the ‘measurement-driven approach’, which “incorporates measures that go beyond user behavior to combine more sophisticated analysis with rudimentary financial indicators”, but which the authors still consider disconnected to the business’ strategic orientation. More recently, Flores (2013) points out that a key distinction is the one between quantitative and qualitative KPIs/metrics. Quantitative metrics have “the objective of counting the measures implemented by digital action”, whereas qualitative ones are assigned to “enhancing the impact of the methods used by a digital campaign”. The author then refers to the ancient AIDA model (awareness, interest, desire, action) to explain which sets of metrics are more adequate to measure the phases of the model.

Return on Investment: One aggregating metric which has deserved widespread recognition for campaign evaluation is the Return on Investment (ROI). Its formula is consensual and applicable to several areas of activity. In marketing, Kotler & Armstrong (2013) define it as “the net return from a marketing investment divided by the costs of the marketing investment”. Duboff (2007) presents it as:

𝑅𝑂𝐼 = 𝐼𝑛𝑐𝑟𝑒𝑚𝑒𝑛𝑡𝑎𝑙 𝑝𝑟𝑜𝑓𝑖𝑡𝑠 𝑝𝑟𝑜𝑑𝑢𝑐𝑒𝑑 𝑏𝑦 𝑡ℎ𝑒 𝑚𝑎𝑟𝑘𝑒𝑡𝑖𝑛𝑔 𝑖𝑛𝑣𝑒𝑠𝑡𝑚𝑒𝑛𝑡 𝐶𝑜𝑠𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑚𝑎𝑟𝑘𝑒𝑡𝑖𝑛𝑔 𝑖𝑛𝑣𝑒𝑠𝑡𝑚𝑒𝑛𝑡

The prevailing conception regarding ROI is one that Kotler & Armstrong (2013) also support: it “can be difficult to measure” since “benefits like advertising impact aren’t really easy to put into dollar returns”, thus being often necessary to take “a leap of faith to come up with a number”. Such challenges are backed up by previous literature, with Eechambadi (2005) referring to a Forrester

(10)

10 study according to which marketing executives’ greatest challenge was “measuring marketing effectiveness or ROI”. An additional issue with ROI – analyzed by authors such as Rust, Ambler, Carpenter, & Kumar (2004) and Mishra & Misra (2012) – is the distinction between short-term and long-term ROI. The first ones bring up ROI’s controversial character, claiming that “Because many marketing expenditures play out over the long-run, short-run ROI is often prejudicial against marketing expenditures” and stating that “The correct usage of ROI measures in marketing requires an analysis of future cash flows”. Kotler & Armstrong (2013) also state that companies are now accounting for “customer-relationship metrics, such as customer satisfaction, retention and equity”, which are “more difficult to measure but capture both current and future performance”.

Measurement challenges: The reasons for firms’ challenges when attempting to measure the return of their DM expenditures are summarized by Bughin, Shenkan, & Singer (2008): “the digital world has developed faster than the tools needed to measure it”, who support their claim on the results of a “McKinsey digital-advertising survey of 340 senior marketing executives”. However, such belief is not consensual, as other sources of literature on the topic have praised that “tools and techniques for tracking performance are emerging rapidly” (Epstein & Yuthas, 2007), fulfilling the potential for good accountability usually attributed to digital means. Even in the beginning of the new century, authors already referred that “the Internet offers unparalleled scope to understand how an online programme has impacted on and influenced a target audience" (Spencer & Giles, 2001). Over the past two decades, in short, theoretical stipulations have been launched in high number but practical advances are still lacking. MEO fits into this spectrum, as one of the firm’s most relevant challenges is the lack of ROI calculation know-how (even for short-term calculations), especially for digital campaigns which have brand awareness instead of sales as the key ultimate objective. Web analytics: As a growingly used tool to face the above-reviewed challenges, Web Analytics (WA) have been another topic of interest for theorists of the marketing field in recent years. Järvinen & Karjaluoto (2015) resort to the WA Association to define it as “the measurement,

(11)

11 collection, analysis and reporting of Internet data for the purposes of understanding and optimizing Web usage”, being able to collect “clickstream data regarding the source of website traffic (…), navigation paths, and the behavior of users during their website visits (…)”. According to Kent, Carr, Husted, & Pop (2011), by using WA “over time, stable website traffic patterns emerge”. However, it is consensual among authors that the use of analytics – from which Google Analytics (GA) has been the most relevant, used and studied tool (Chaffey & Smith, 2008)(Tonkin et al., 2010) – is not enough by itself, and “understanding the reason why something is happening on a website” is a crucial managerial skill to complement analytical data (Kent et al., 2011). A report from IBM Software (2012) on the merits of big data analytics for DM brings up companies’ need of treating data as a high-value, high-impact strategic asset, as well as addressing the “talent gap” in staff’s digital and technological skills, in order to follow firms’ “commitment to continuous performance measurement”. Identifying this skills gap goes in line with the research topic of Royle & Laing (2014), when addressing the issues of DM practitioners on the telecom industry.

It is clear that proper data analysis is a key driver of marketing performance measurement and that WA emerge as a powerful tool to collect, measure and report it. Optimizing MEO’s use of WA to increase accountability and decrease data fragmentation was one of this WP’s key goals. Overall, this project’s aims at creating value by adding to the knowledge on performance measurement in the digital marketing field, as well as by suppressing some of MEO’s gaps and expressed needs when addressing the topics covered on this literature review exercise.

4. Methodology

As this WP was a collective one throughout most of its duration, this section will majorly place incidence on the methods adopted by the students’ team and fewer words will be dedicated to specific individual procedures, as those were only predominant on the final weeks of the project. After the kick-off meeting with the client on September 16, 2015, some tools and frameworks from typical consultancy practices were adopted from the first day of work (September 21) on, according

(12)

12 to the Field Lab’s stipulations, as a way of providing a clear structure for the work ahead. The main framework used consisted of four main phases: 1) Problem definition; 2) Problem structuring; 3) Data gathering and analysis; 4) Recommendations’ elaboration and delivery.

Therefore, the first key priority was to clearly define the broad strategic problem to address, as well as the correspondent research problem (problem definition phase), in order to then develop a diagram/“issue tree” (problem structuring phase) with a list of possible solutions and specific actions to undertake for each of those solutions. On that first day, a meeting with the team’s liaison – DM Process Improvement Manager at MEO, from the team with which this collaboration focused on (“Direção Digital – Campanhas B2C” (DDC)) – was held. Based on: a) stipulations made by MEO on the kick-off meeting, b) the inputs about MEO and its DM activity from the meeting with the liaison and c) the secondary data research that followed, the broad problem to address was defined: “How to increase MEO’s reach efficiency and effectiveness of DM campaigns?”. A second reunion with the liaison on October 6, a steering meeting with the WP’s advisor on October 13 and a meeting with MEO’s Head of DM on October 14 were instrumental to narrow down the list of paths of action to follow. From then on, two key methodology aspects arose:

 From the 16 lines of action (corresponding to 4 solutions) the team had initially come up with (Appendix 2), only 6 were maintained for further analysis: implementation of programmatic buying, adoption of a standardized metric for campaign evaluation (which later ended up turning into the topic of this individual thesis), increase of MEO' digital ads’ attractiveness and of its cross-media interactions (related to taking advantage of new online trends). The implementation of advanced targeting techniques was also left as a possible path. The rationale for the optimization of the “issue tree” included 3 main factors: the client’s list of priorities, the team’s time constraints and the team’s know-how and predicted learning curve for the different topics on the table.

 For these 6 final issues, the team defined the specific research methods to use (data gathering phase). In order to gather information on the most different topics from internal sources,

(13)

13 in-depth interviews (qualitative research method) were carried out with specifically selected MEO staff (based on their relationship with the WP’s goals). For that effect, on October 21 the group interviewed MEO’s Social Media manager and, on October 22, a DM Manager from the DDC team. Finally, on October 29, the interviewee was a DM manager from the E-Commerce team (PT Empresas segment). The main goal of these interviews was the setup of an environment that would (and did) lead to an unstructured and free talk about the relationship of each department with the development of a digital campaign at MEO, as well as the possibility to uncover underlying feelings about internal factors. All the interviews took place at MEO’s office.

Additionally, since some of the issues which the team had to address also depended on the exterior of the company (most notably ad attractiveness and cross-media interaction and online habits), two focus groups were organized with MEO customers (qualitative research procedure: structure draft in Appendix 3). Both focus groups counted with 9 previously screened elements between the ages of 21 and 53 (11 men and 7 women) and lasted for around 90 minutes, having been carried out on October 27 and 28 at a closed environment at Nova SBE. The main goal of these sessions was to obtain feedback on different topics of interest for the WP, by promoting unstructured free-flowing sessions with guidance towards the team’s goals. The interviews and the focus groups were audio-recorded to facilitate subsequent data analysis. Finally, in order to quantitatively test the hypotheses which emerged from the focus groups’ conclusions, an online survey (quantitative research method: questions in Appendix 4) was launched and answered by a total of 267 people (average age of respondents was 23, with 43% of them being female and 57% being male), with no screening of who filled it out. Its online launch envisioned ensuring the highest number of answers in a time constraint scenario (it ran for 5 days, from November 15 to 19). Four main types of scales were used: multiple choice, constant sum scaling, rank order and likert scaling.

Data from these research methods had to be analyzed to generate valid conclusions as a basis for recommendations. The content of the interviews was transcribed into an analysis grid, gathering

(14)

14 information on selected topics of interest. For the focus groups, the analysis was similar: a grid was created with the participants’ contributions on selected topics, leading to final conclusions and to the generation of hypotheses. Key topics were: MEO’s online and offline reach effectiveness, brand recall, cross-media interaction (from the customer’s side) and online campaigns’ attractiveness. The survey’s analysis was, thus, more applied to the WP’s priorities and carried out using SPSS and Microsoft Excel. It involved descriptive and frequencies’ statistical analyses – using stats such as mean, standard deviation, kurtosis and skewness – and cross-tab analyses by testing relevant variables’ statistical dependency (ANOVA and chi-square testing). In short, the methods for each set of issues (reflecting the division of themes for the team members’ individual reports) were:

a) “Implementation of programmatic buying” and “Implementation of advanced targeting techniques”: i) secondary data research (papers, external reports and websites) in order to gather information on best practices; ii) cross-tab analysis from the survey to create a more specific target profile to present to the client, which can have impact on both of these topics.

b) “Increase campaign attractiveness” and “Take advantage of new online trends” (including “cross-media interaction”): i) secondary data research to gather information on new entrepreneurial practices, new rich media ad formats and consumer habits; ii) descriptive statistics and cross-tab analysis supported by Chi-Square tests of the survey’s results, in order to obtain data about habits and consumer preferences regarding online advertising.

c) “Adopt a standardized metric for campaign evaluation” (which originated this thesis’ topic): i) secondary data research (papers, external reports and websites) in order to get to know state-of-the-art practices and findings regarding DM campaigns’ return evaluation; ii) immersion into the platforms used by MEO (MDX and GA); iii) complementary training through completing a GA introductory course; iv) use of the inputs from the in-depth interviews (relevant parts of the analysis grid: Appendix 5); v) analysis, treatment and optimization of MEO campaigns’ data.

(15)

15

5. Discussion of findings/As-Is analysis

The key findings regarding the author’s specialization issue for this WP are an outcome of the methods described on paragraph c) above, and allow for greater understanding of MEO’s current position (“As-Is” analysis). Due to the topic’s nature (with a sole internal component and no direct external influence), this section has been named “Discussion of findings” instead of “Results”, as the findings were mostly obtained from secondary data and not from primary research methods (from these, only the content and conclusions from the in-depth interviews were used for the topic). Adding to the DM roadmap presented by the company at the kick-off meeting – according to which campaign management, analysis and reporting are a key priority for future optimization – the first meeting with the team’s liaison revealed that, until 2015, display campaigns’ reporting was significantly incomplete and that most analyses of campaign performance were based on cost metrics, with little to none focus on website engagement and campaign returns. On a subsequent meeting with the same person, the team was informed that cost metrics such as Cost per View (CPV), Cost per Click (CPC) and Cost per Mille (CPM) were the main ones used by MEO for campaign performance analysis, along with ad engagement metrics such as Views (for video ads) and Sessions to the website. A ROI analysis was planned to be a reality at the time, but data fragmentation (between different teams within the company and between different and non-interrelated web platforms) and data insufficiency (the media agency working with MEO often does not input the necessary values on the ad serving platform used by the company - MDX) were listed as key obstacles that have not allowed for it to be carried out until the beginning of the WP. Besides that, on the meeting held on October 14 with the manager of the DDC department, another obstacle was mentioned: MEO’s difficulty in defining what ROI effectively is for the department’ campaigns, due to the fact that their most immediate goal is to build brand awareness and not so much to directly have an impact on the company’s sales, leading to difficulties in quantifying the effective returns of each campaign. Kireyev, Pauwels, & Gupta (2013) describe the difference of

(16)

16 roles of display and search advertising along the conversion funnel: “A consumer may be exposed to a brand through display ads, she may click on these ads to get more information, and may eventually convert. This is the direct impact of display ads on conversion that most studies find to be very small. Alternatively, a consumer could be actively searching for a product online, where she encounters a search ad, clicks on it, and converts.” Finally, some lack of coordination between teams regarding campaign reporting and analysis was referred transversally as a problem (e.g.: leading to members of the DDC department not being aware of some features that the Web Analytics team had already developed in order to make campaign evaluation easier for the future). As it was mentioned on the “Methodology” section, three in-depth interviews were carried out with MEO elements. Regarding the issue on which this report focuses, relevant insights and information arose from the interviews and fit into this “As-Is” analysis/discussion of findings part:

 The DDC operates at the top of the conversion funnel (Appendix 6), focusing on brand building and using a larger set of tools, channels and platforms than the E-Commerce team, whose work is mainly with Google AdWords and keyword optimization (with the direct goal of converting/selling/monetizing). Since AdWords is singlehandedly managed through GA, the E-Commerce team can easier track the cost and direct return of its campaigns (which are mostly ongoing throughout the year). Conversely, the DDC uses an ad serving platform (Sizmek MDX, managed by MEO’s media agency, OMD, at the ad level) and also GA for more accurate reporting on the firm’s website activity, with data being often inconsistent between the two platforms.

 The interviewed member from the DDC went on to further explain the key difference between the team’s main objective and E-Commerce’s: “E-Commerce focuses almost exclusively on a cost per adhesion/ conversion. (…) In our case, in branding, our most relevant cost metric ends up being CPM (…) or eventually the cost per contact (…) regardless of that having a certain cost of conversion (…)”. What was apprehended from then on was essentially the ill-coordination between all of MEO’s DM teams and, above all else, a certain lack of focus, from the DDC team,

(17)

17 on the end result of the whole marketing activity (the final conversion), instead concentrating solely on the intermediary process (branding) without greatly envisioning end-of-the-funnel success.

 The same person also provided insights regarding the DDC’s key goal regarding performance measurement of display campaigns, stating that “[what we are working on is] that the measuring becomes so reliable that we can measure, end-to-end, what happens in a campaign”. From then on, the author began specializing on the topic, and a three-headed process was set from then on: i) further concrete learning about WA (by taking Google’s online course “Digital Analytics Fundamentals”); ii) immersion into the platforms used by MEO; iii) gathering, treatment and analysis of past MEO campaigns’ data to find optimization opportunities, develop relevant metrics and a concrete way of calculating MEO’s branding campaigns’ ROI. The most relevant findings from such approach, with deep impact on the following section of this report, were as follows:

a) From all of MEO’s display campaigns from 2015 (when the “Digital Transformation” process began at the firm), only 7 met the criteria to be considered for analysis (having enough data

records on MDX and GA): “Primavera”, “LG G4”, “Verão”, “Verão SELO”, “Rentrée 1ª Fase”, “Samsung Surf” and “Rentrée 2ª Fase”. Despite the key branding focus of the campaigns, all of

them aimed at promoting a product or service, such as Cable TV, Mobile Internet or Smartphones. b) For all campaigns, some pieces of information were missing or incoherent between the two platforms (e.g.: website sessions resultant from Facebook ads registered on GA, without any record of Facebook ad impressions on MDX), pointing out to the issues of data fragmentation and insufficiency. Therefore, only common data between both platforms was considered.

c) From the analysis, it was concluded that the relevant campaign KPIs that MEO could directly access from both platforms were: Total Media Cost, Impressions, Average Frequency, Unique Impressions, CPM, Sessions, Bounce Rate, Time on Site, Pages per Visit, number of E-Commerce transactions and goal completions (explanation of these metrics: Appendix 7).

(18)

18 d) Relevant metrics of campaign return could not be directly accessed, partly due to the misuse of the GA platform by the DDC, according to information that was only provided on December 2 by one of MEO’s WA specialists. Effectively, goals are correctly defined within the property www.meo.pt (MEO’s website) on its GA account, and most of them have a monetary value associated ((Tonkin et al., 2010) cover the concept of goal value), but the DDC did not use such tool. The online course taken revealed the lack of retroactivity of such situation, impeding the obtainment of those goal values for past branding campaigns in a non-manual way. MEO’s website conversions are divided into non-E-Commerce (Click-to-call buttons, Forms filled, Smartphone acquisition with loyalty points and Smartphone acquisition by instalments) and E-Commerce ones. e) In terms of campaign attribution – “(…) dividing up the value of an online sale (or conversion) and distributing fractions of that value across the different touchpoints that led to the sale” (Google, 2014) – MEO uses the last-click attribution model, which is the default one on GA but has the major flaw of not reflecting “the role of the mix of media such as display advertising and natural search in influencing sales” (Chaffey & Smith, 2008). According to the same authors and other literature, “a common phenomenon in online advertising is the display advertising halo effect where display ads indirectly influence sales (…) known as ‘view-throughs’ or post impression effects” and the sole use of the last-click attribution model does not account for that effect. Effectively, MEO does not use any method to calculate those view-through conversions (which is possible to do, as it will be explained on the following section of the report), therefore significantly undervaluing the performance of its display campaigns.

f) Ad tagging is a feature that “can be used to track all [of a company’s] marketing and advertising”. On GA, the key ad tags for display advertising are: medium (“designed to work as the broadest or highest-level dimension for campaign tracking”), source (“the place the visitor responding to the ad comes from”), campaign and ad content (Tonkin et al., 2010). At MEO, the tagging practice requires optimization, as there are situations in which the feature has been wrongly

(19)

19 used (e.g.: different teams tagging similar ads in a different way; some ad contents being placed on the medium tag section). This leads to serious campaign tracking issues, making it hard to have an accurate idea of the impact of separate sources, mediums and campaigns developed by MEO.

g) Several other Web Analytics features are being sub-used or not used at all by the department and not contributing for the optimization of performance measurement as they could.

6. Main contribution and additional recommendations

The above-described “As-Is” situation led to the development of the most impactful contributions of this WP. Those contributions and the associated recommendations have been divided into two main parts. The first one (“Main contribution/Structural change”) focuses on the work developed by the author in terms of effectively finding a way of measuring the performance of MEO’s DDC campaigns, whereas the second one (“Additional recommendations/Quick wins”) comprises a list of recommendations the firm should adopt to further optimize its measurement capacity.

6.1. Main contribution/Structural change

Based on the findings analyzed on the previous section of this WP, the author’s key objective was clearly defined: to develop work that would allow the DDC team to measure its campaigns’ return, with a key focus on a ROI analysis that would allow to compare the performance of different campaigns and deduce insights from it. The work’s structure and main outcomes were as follows:  A table was created using MS Excel in order to manually gather data on the campaigns listed on the findings section of this report (Appendix 8), with the end goal of creating a visually intuitive document for campaign reporting and analysis. For each campaign, a list of the channels in which MEO advertised was also included, with the subsequent directly accessible data (Ad description, Cost and Ad Engagement metrics) for each one and also data manually calculated (some of the Website Engagement metrics) (a sample example of the channels’ list: Appendix 9).  The table was divided into 6 sections of columns: 1. Campaign and ad description; 2. Costs; 3. Ad Engagement; 4. Website Engagement; 5. Revenues; 6. Returns, according to: secondary data analysis; directly accessible KPI values; non-directly available but calculated data. The aim of such

(20)

20 division was to promote campaign end-to-end measurement, by starting to focus on revenues and returns (non-contemplated components until then) besides Costs, Impressions, Sessions and Views.  Regarding the metrics used on the table, the ones mentioned on section c) of page 17 of this report were included. Besides those, several new ones were inserted and calculated by the author, as they were considered to be of added value for the WP’s goals. Those metrics (their way of calculation and rationale behind their use: Appendix 10) were: Sessions/Impressions ratio; Cost per Visit; Engaged Visit %; Cost per Engaged Visit; Real conversion rates; Direct CPA; View-through conversions (VTCs); Total campaign revenue; RPM; Direct ROAS; Campaign ROI.

 In order to calculate a campaign’s ROI, cost and revenue metrics would have been enough. However, it was not the WP’s intention to move from a “cost and engagement” centric-approach to a merely return-centric one. Therefore, ad and website engagement metrics were also included and Excel’s conditional formatting tool was applied to them, so that their values for each campaign could be automatically compared and intuitively analyzed against display ads benchmarks from the worldwide telecom industry and against global MEO website benchmarks, using the Benchmark tool on GA, for the year of 2015. (Appendix 11: Table with those benchmark values).  As for the calculation of each campaign’s Return on Investment, which was always the predominant goal of the work that was carried out, the included components were five:

A. Total media cost (obtainable on MDX); B. Value of E-Commerce transactions attributed to the campaign using the default last-click attribution (obtainable on GA); C. Number of non-E-Commerce goal conversions attributed to the campaign with last-click attribution (obtainable on GA); D. Value of those non-E-Commerce goal conversions (the correct values had to be manually calculated using an assistant table on Microsoft Excel, listing each type of conversion and the values which have been recently set up by MEO’s WA specialists (Appendix 12)); E. Estimated value (as a temporary solution) of view-through conversions (subject to future optimization).

(21)

21 𝑅𝑂𝐼 =(𝐵 + 𝐷 + 𝐸) − 𝐴

𝐴

 Component E. was the most difficult to obtain and it will have to be subject, as mentioned, to further optimization by MEO, as it is impossible to calculate the right number and value of each campaign’s VTCs with the information currently held by the firm. MDX elaborates pre-defined conversion reports in which VTCs are included, but website goals are not correctly defined on that platform, as of January 2016. Therefore, goal conversions (whether post-click or view-through ones) on MDX are irrelevant for now, with the difference that post-click ones are accessible on GA (and correctly) and view-through ones are not, making it impossible to obtain valid information for those before setting up the right goals on MDX. Besides that, considering VTCs for ROI calculations might overvalue a campaign’s returns, since “it is entirely possible to reward display ads for view-through visits or conversions that the ad really had nothing to do with” (Tonkin et al., 2010). Employing a method to determine which % of VTCs must be attributed to each campaign is thus fundamental, but also impossible to do with the data MEO can currently access.

The temporary solution found by the author consists in using assumptions for an average “post-click conversions to VTCs ratio” (1:20 in this case (Zalman, 2010)) and for the % of VTCs that should be attributed to a campaign on average (the author assumed the value of 17,5% in this case, as (AdRoll, 2014) claims that most marketers attribute 10-25% of VTCs), based on internet articles and reports – due to the absence of more relevant benchmark information. These assumptions are far from accurate (their drawbacks: Appendix 13) but that is not seen as a major issue since they only mean to demonstrate how VTCs can be calculated with practical numbers which allow for effective ROI calculation, even if not totally reliable for now. The ultimate goal is for MEO to absorb the ROI formula and optimize the calculation of VTCs by following these steps:

1. Correct (or coordinate with OMD to do it) goal setting on MDX, and attribute to each goal the same value that they have been attributed on GA. From that point on, obtain valid VTCs value

(22)

22 (and rate) data for each campaign, on MDX. The standard lookback window of 30 days on the platform may be maintained or changed according to the firm’s insights and own experience.

2. To avoid the overvaluation of a campaign’s performance, MEO must determine the % of VTCs to attribute each campaign, by running A/B testing using Public Service Announcements (PSAs): “the advertiser pays for display campaigns for selected non-profit organizations (…) to determine the lift in performance between users who were exposed to your ads and those who were not. The advertiser gets data, the non-profit gets free advertising.” (Morimoto, 2013). This involves dividing a homogeneous audience into a test (exposed) and a control (not exposed) group to determine the lift in performance between both groups. The “normal % split is 90/10 (test/control)”, according to the same article. The final formula should be as follows (Galyon, 2014):

𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑 𝑉𝑇𝐶 % = 𝐵𝑟𝑎𝑛𝑑𝑒𝑑 𝑐𝑎𝑚𝑝𝑎𝑖𝑔𝑛 𝑉𝑇𝐶 % − 𝑃𝑆𝐴 𝑐𝑎𝑚𝑝𝑎𝑖𝑔𝑛 𝑉𝑇𝐶 % 𝐵𝑟𝑎𝑛𝑑𝑒𝑑 𝑐𝑎𝑚𝑝𝑎𝑖𝑔𝑛 𝑉𝑇𝐶 %

3. If the solution presented on point 2. above is regarded as being too complex or difficult to carry out for a given campaign (e.g.: due to short campaign lifetime, since PSA testing must be held for at least the same amount of time as the lookback window), MEO may instead simply reduce the lookback window on MDX from 30 days to 1, 2 or 3 days and this way attribute 100% of the obtained VTCs to the campaign. This is a less reliable method, since there is no guarantee of causality between the campaign and the conversion – even with the reduced lookback window – but it might be a quick path to obtain a proxy value when there are limitations to PSA testing.

To sum up, this whole line of work aimed essentially at adding value for MEO on 5 fronts: i. Match Cost and Ad Engagement data from Sizmek MDX with Website Engagement and Return metrics from GA, for each campaign and channel, in order to develop an end-to-end campaign performance measurement method, including financial and non-financial outcomes;

(23)

23 ii. Suppress internal communication gaps and provide clarification to the DDC team on key issues (concretely the fact that it is possible to effectively track the direct last-click conversions of each display campaign and attribute them their respective value, contrarily to what was thought);

iii. Include into the conversation the topic of benchmark analysis when evaluating the performance of each campaign’s and channel’s ad and website engagement metrics;

iv. Come up with a (despite not perfectly accurate, as explained) way of accounting for the display advertising halo effect (indirect effect on sales) mentioned by Chaffey & Smith (2008);

v. Aggregate all these factors into a ROI analysis that was not performed due to the absence of the contributions i., ii. and iv. above. The developed ROI formula allows for customization, as well as for campaign isolated and comparative analyses. Although this WP’s key goal was never the specific analysis of the 7 sample campaigns, the delivered table effectively allows for a more complete end-to-end analysis of those 7 campaigns than what was possible and done before.

6.2. Additional recommendations/“Quick wins”

Adding to the main contribution, complementary recommendations were developed, based on internal and external data and focusing on further enhancing the DDC’s measurement capabilities: 1. Define a concrete Web Analytics Measurement Plan (WAMP) in order to eventually slightly refine the delivered measurement table. In this context, the WAMP must comprise essentially 4 stages and be discussed with top management, including: identifying the actual business objectives that WA attempts to address; identifying goals for each objective; listing KPIs for each goal; creating targets for each KPI. (Tonkin et al., 2010)(Kaushik, 2015a)

2. Still related to the Measurement Plan, it would be especially advisable for the department to periodically adjust the target/benchmark values for the ad and website engagement metrics on the table (Appendix 11), based on internal knowledge, experience and business objectives.

3. Regarding campaign tracking, conversions and organization in general, it is fundamental to start correctly (and coherently) tagging the department’s ads on GA. Three changes are a priority:

(24)

24 a. Use one tag per campaign (avoid situations such as e.g.: having the same campaign listed as both “meoprimaveratablet” and “meoprimaveratablet~”, that lead to data fragmentation); b. Start tagging ad content, as it is one of the 4 most important ad tags and the department currently does not do it at all, instead inserting the ad content on the “medium” section; c. Precisely, stop tagging the “medium” section with ad content descriptions. Define only 2 or 3 medium tags – “display” and “rich media” would be enough. This will allow the team to explore several GA functionalities, such as accurately analyzing the changes of display ads’ conversion performance between distinct attribution models (see Recommendation 5. below).

4. Also a conversion-related issue, the definition of Micro Conversions is another recommendation. Currently, MEO only has Macro Conversions (Click-to-call, Smartphone sales with loyalty points or with instalments, Forms filled) set as goals on GA (besides E-Commerce transactions, which are automatically tracked). However, there are numerous website sessions which generate value for the company but do not convert into any of those events (especially with display ads, which have a lower than average conversion rate). Tracking those additional goals (called Micro Conversions since they are indicators of website success, despite not being the main goal of the business (Kaushik, 2015b)) will provide a clearer image of a campaign’s overall performance, by accounting for other factors than sales and direct conversions. Considering the configuration of MEO’s website, some potentially relevant Micro Conversions for it to track would be: access to the store locator page; e-mail sign-ups on the website and clicks on the print button on the few pages in which that functionality exists. Attributing a value to these Micro Conversions in the future, based on business projections and insights, would be of added utility for the company. 5. Change the used attribution model on GA, moving towards one that accounts for the impact of more than one single touchpoint along the customer journey, such as linear attribution (same value attributed to all touchpoints) or time decay (more value attributed to the final ones). This needs to be complemented with correct tagging (Recommendation 3. above) and with correct goal

(25)

25 and goal value setting, in order to allow for full model functioning. Alternatively, MEO can also customize an attribution model based on one of these standard models, according to its own needs and specificities (Kaushik, 2015d). The new attribution model will, in theory, be more suited than last-click in terms of measuring the impact of display ads on post-click sessions and conversions, serving as a better complement to the display VTCs that will begin to be measured by the firm.

6. Implement two other relevant WA tools, attending to MEO’s specific case:

a. Develop A/B, multivariate or A/B/N testing for different landing page features. The analyzed campaigns showed high bounce rates (an average of 66,05%), and bad landing page configuration or misleading connection between the ad content and the landing page are some of the reasons that can justify it. Using a tool such as GA Content Experiments will allow the company to test up to 10 variations of one landing page (A/B/N testing) and to draw strategic conclusions.

b. Use Custom Reports (Tonkin et al., 2010) in order to customize data visualization to the needs and context of the company. The DDC does not use the functionality, but it should not be overlooked for a department attempting to become more analytically-minded. Effectively, for future campaigns – if tagging starts being correctly performed, if goal values are correctly adjusted by the team and if enough hours of work are dedicated to the topic of performance measurement – the adequate configuration of custom reports (by choosing the correct dimensions and metrics) might be the exact tool to automatize the process of data convergence (with information from MDX) that the author had to develop manually due to the absence of those conditions.

7. Limitations and future research

As a final note, despite the belief that the project yielded satisfactory results for the client’s future path along its digital marketing roadmap, the author believes that even more detailed and perhaps more reliable outputs could have been produced if it had not been for some of the limitations faced during the project, at two key levels. At the project level, the most noticeable were the lack of a more concrete integration of the WP on MEO’s structure, possibly related to the internal turmoil

(26)

26 and changes that it is going through. At the information level, the key obstacles were analytical data insufficiency to promote a more robust analysis, as well as data fragmentation/inconsistency between the used WA platforms and information contradictions among company members, which led to some forced delays and changes of plans on the last two weeks of the project.

Effectively, some topics of potential interest for the company were left out of the project’s scope but might be of significance for future research and eventual application by MEO. Among those are: i) developing attempts to measure or accurately estimate the impact of online advertising on offline conversions and ii) the impact of offline advertising on online conversions, which Tonkin et al. (2010) address; iii) using WA tools for segmentation and analysis of behaviors and performances of groups of customers, which is still a ill-used potentiality by the company; iv) analyzing the new digital analytics concept of “viewability”, which accounts for impressions that have effectively been seen and not only served. Sizmek gained official accreditation for its viewability solution in 2013 (Sizmek, 2014). MEO still uses served impressions only.

Reference List

AdRoll. (2014). Attribution modeling for data-driven marketers.

Alves, I. F. (2015). Meo mantém liderança nas comunicações móveis e Nos aumenta quota de mercado. Jornal de Negócios. Retrieved from

http://www.jornaldenegocios.pt/empresas/detalhe/servicos_moveis_meo_mantem_a_lideran ca_mas_nos_aumenta_quota_de_mercado.html

ANACOM. (2015). Serviço de Televisão por Subscrição - Informação Estatística (2o_Trimestre

de 2015).

Banfi, F., Caylar, P.-L., Duncan, E., & Kajii, K. (2013). E-journey: Digital marketing and the “path to purchase.” McKinsey & Company - A Publication of the Telecommunications, Media, and Technology Practice, (22).

Bughin, J., Shenkan, A. G., & Singer, M. (2008). How poor metrics undermine digital marketing. The McKinsey Quarterly, (October), 1–5.

Carat. (2015). Carat Ad Spend Report.

Chaffey, D., & Smith, P. (2008). eMarketing eXcellence - Planning and optimizing your digital marketing (3rd Ed.). Elsevier. http://doi.org/10.1007/s13398-014-0173-7.2

(27)

27 Duboff, R. S. (2007). ROI for Marketing: Balancing Accountability with Long-Term Needs. Eechambadi, N. (2005). The Measure of Marketing - From Measurement to Improvement. In

High Performance Marketing: Bringing Method to the Madness of Marketing (pp. 63–96). Epstein, M. J., & Yuthas, K. (2007). Evaluating the Effectiveness of Internet Marketing

Initiatives.

Flores, L. (2013). The different types of metrics and KPIs available: “quantitative” vs. “qualitative.” In How to Measure Digital Marketing: Metrics for Assessing Impact and Designing Success (pp. 47–50).

Galyon, C. (2014). Valuing View-Through Conversions. Retrieved from http://www.swellpath.com/2014/01/valuing-view-conversions/ Google. (2014). Measure What Matters Most. Think with Google.

IBM Software. (2012). Moving up the digital marketing maturity with big data analytics. Jackson, S. (2009). Cult of Analytics: Driving online marketing strategies using web analytics.

Elsevier. http://doi.org/10.1016/B978-1-85617-611-8.00016-X

Järvinen, J., & Karjaluoto, H. (2015). The use of Web analytics for digital marketing performance measurement. Industrial Marketing Management, 50, 117–127. http://doi.org/10.1016/j.indmarman.2015.04.009

Kaushik, A. (2015a). Digital Marketing and Measurement Model. Retrieved December 7, 2015, from http://www.kaushik.net/avinash/digital-marketing-and-measurement-model/

Kaushik, A. (2015b). Measure Micro and Macro Conversions. Retrieved from

http://www.kaushik.net/avinash/excellent-analytics-tip-13-measure-macro-and-micro-conversions/

Kaushik, A. (2015c). Measure the Real Conversion Rate. Retrieved from

http://www.kaushik.net/avinash/excellent-analytics-tip-8-measure-the-real-conversion-rat Kaushik, A. (2015d). Multi-Channel Attribution Modeling: The Good, Bad and Ugly Models. Retrieved from http://www.kaushik.net/avinash/multi-channel-attribution-modeling-good-bad-ugly-models/

Kent, M. L., Carr, B. J., Husted, R. A., & Pop, R. A. (2011). Learning web analytics: A tool for strategic communication. Public Relations Review, 37(5), 536–543.

http://doi.org/10.1016/j.pubrev.2011.09.011

Kireyev, P., Pauwels, K., & Gupta, S. (2013). Do Display Ads Influence Search? Attribution and Dynamics in Online Advertising. Harvard Business School Working Paper, 1–30. Kotler, P., & Armstrong, G. (2011). Principles of Marketing. (Pearson Prentice Hall, Ed.) (14th

Ed.). http://doi.org/10.2307/1250103

(28)

28 Success.

Mishra, S., & Misra, M. (2012). An Assessment of ROI in Online Perspective Leading to a Critical Review and Framework Development Towards Standardization of ROI Practices. Journal of Internet Banking and Commerce, 17(1), 1–18.

Morimoto, J. (2013). Tips for running PSA (Test and Control) campaigns. Retrieved from http://3qdigital.com/display/tips-running-psa-test-control-campaigns/

O’Sullivan, D., Abela, A. V., & Hutchinson, M. (2009). Marketing performance measurement and firm performance. European Journal of Marketing, 43(5/6), 843–862.

http://doi.org/10.1108/03090560910947070

Ribeiro, S. (2015). Meo lidera serviços de acesso à internet fixa e móvel. Jornal de Negócios. Retrieved from

http://www.jornaldenegocios.pt/empresas/detalhe/meo_lidera_servicos_de_acesso_a_interne t_fixa_e_movel.html

Royle, J., & Laing, A. (2014). The digital marketing skills gap: Developing a Digital Marketer Model for the communication industries. International Journal of Information Management, 34(2), 65–73. http://doi.org/10.1016/j.ijinfomgt.2013.11.008

Rust, R. T., Ambler, T., Carpenter, G. S., & Kumar, V. (2004). Measuring Marketing Productivity: Current Knowledge and Future Directions. Journal of Marketing, (October), 76–89.

Ryan, D., & Jones, C. (2009). Understanding Digital Marketing - Marketing strategies for engaging the digital generation.

Ryan, D., & Jones, C. (2011). The Best Digital Marketing Campaigns in the World: Mastering the Art of Customer Engagement.

Sizmek. (2014). Sizmek Benchmarks Report - 2014.

Spencer, C., & Giles, N. (2001). The planning, implementation and evaluation of an online marketing campaign. Journal of Communication Management, 5(3), 287–299.

http://doi.org/10.1108/13632540110806839

Tonkin, S., Whitmore, C., & Cutroni, J. (2010). Performance Marketing with Google Analytics: Strategies and Techniques for Maximizing Online ROI.

Valos, M. J., Ewing, M. T., & Powell, I. H. (2010). Practitioner prognostications on the future of online marketing. Journal of Marketing Management, 26(3-4), 361–376.

http://doi.org/10.1080/02672571003594762

Zalman, E. J. (2010). Why view-through is a key component of campaign ROI. Retrieved from http://www.mobilemarketer.com/cms/opinion/columns/7367.html