Group environments that require the support of GSS can be classified by the time of the meetings, either synchronous (same time) or asynchronous (different times), or the location of the group (either face to face or dispersed) (DeSanctis and Gallupe, 1985). In the early 1980s, GSS research initially focused on “decision rooms” (synchronous and face to face) such as those facilities established at the University of Arizona and the University of Minnesota.
Negotiation support systems (NSS) also operate in a group context but as the name suggests they involve the application of computer technologies to facilitate negotiations (Rangaswamy and Shell, 1997). As GSS were developed, the need to provide electronic support for groups involved in negotiation problems and processes evolved as a focused sub-branch of GSS with different conceptual foundations to support those needs. It should be noted that this research is focused on a major subset, but not all, of the research that appears in the Group Decision and Negotiation journal and conference, namely papers that mention a system in investigation of issues in group decision and negotiation.
Two approaches to constructing systems that support negotiation arose, namely problem oriented and process oriented (Jelassi et al., 1990). Problem-oriented NSS products include Co-oP (Bui and Jarke, 1986), DECISION MAKER (Fraser and Hippel, 1984), GDSI and MEDIATOR (Kersten, 1987). These problem- oriented systems focus on providing support to support negotiation for specific problem types. On the other hand, process-oriented NSS focus on providing general support of the give-and-take process of negotiation (Chaudhury, 1995).
Negotiation support systems has its conceptual foundations in game theory (Von Neumann and Morgenstern, 1947), which supports many models of bargaining (Raiffa, 1982), and social choice theory (Arrow, 1951). While NSS may be considered a branch of GSS research, it has evolved using different theories, technologies and applications and now stands as a significant branch of decision support history.
Methodology and Design
In order to explore the concerns expressed in Section 2 above, the following general research questions were formulated in order to guide the project:
What publication and funding trends have occurred in GSS and NSS research?
What type and quality of research is being done?
How relevant is GSS and NSS research?
To answer these questions this project involved the bibliometric content analysis of relevant published research.
Time Frame
The time period of published research chosen for this project is 1990 to 2010. While GSS and NSS began to evolve in the late 1970s and early 1980s, it was not until the end of the 1980s that the field could be regarded as relatively stable and mature. Further, the period since 1990 also marks a period in information systems discipline which saw a significant growth in the use of non-positivist research methods. In industry, the analysis period saw the deployment of several new generations of collaboration technologies. To reflect these generations, the sample has been divided into three time periods: 1990 to 1996, 1997 to 2003, and 2004 to 2010.
The Article Sample
The sample of articles analysed is GSS and NSS research published between 1990 and 2010 in 16 journals.
These included the field’s two major Decision Science journals Decision Sciences (DS) and Management Science (MS). Then three specialist DSS journals were included: Decision Support Systems (DSS), Journal of Organisational Computing & Electronic Commerce (JOC&EC), and Group Decision & Negotiation (GD&N). The remaining nine journals were the major Information Systems journals Accounting, Management & Information Technologies/Information & Organization (I&O), Communications of the Association for Information Systems (CAIS), European Journal of Information Systems (EJIS), Information & Management (I&M), Information Systems Journal (ISJ), Information Systems Research (ISR), Journal of Information Technology (JIT), Journal of Management Information Systems (JMIS),
Journal of Strategic Information Systems (JSIS), Journal of the Association for Information Systems (JAIS), and MIS Quarterly (MISQ). Tables 1 and 2 show GSS and NSS publication where these journals have published any papers of that type.
Table 1: GSS Papers by Journal
Journal 1990 - 1996 1997 - 2003 2004 - 2010 Total No. % of
Period No. % of
Period No. % of Period No. % of Sample
I&O 2 1.3 3 2.1 1 1.2 6 1.6
DS 3 1.9 1 0.7 0 0.0 4 1.0
DSS 28 17.8 27 19.1 24 28.2 79 20.6
EJIS 2 1.3 4 2.8 0 0.0 6 1.6
I&M 14 8.9 16 11.3 4 4.7 34 8.9
ISJ 1 0.6 3 2.1 1 1.2 5 1.3
ISR 6 3.8 4 2.8 1 1.2 11 2.9
JAIS 0 0.0 0 0.0 4 4.7 4 1.0
JIT 2 1.3 2 1.4 1 1.2 5 1.3
JMIS 21 13.4 23 16.3 7 8.2 51 13.3
JOC&EC 37 23.6 14 9.9 2 2.4 53 13.8
JSIS 0 0.0 0 0.0 0 0.0 0 0.0
GD&N 26 16.6 39 27.7 39 45.9 104 27.2
MS 6 3.8 1 0.7 0 0.0 7 1.8
MISQ 9 5.7 4 2.8 1 1.2 14 3.7
Total 157 100.0 141 100.0 85 100.0 383 100.0 The publication distribution of GSS articles is shown in Table 1. The Journal of Group Decision and Negotiation (GD&N), which is the primary specialist journal for the GSS discipline, is GSS’s leading publisher with 27.2% of GSS articles overall and this share has increased from 16.6% to 27.7% to 45.9%.
GD&N is clearly the leading outlet for GSS research and is becoming more so. Other major outlets for GSS research are DSS with 20.6%, JOC&EC with 13.8% and JMIS with 13.3%. JMIS and I&M are the only general IS journals that have published GSS papers in substantial numbers, but this has (alarmingly) seen a significant fall over time. Most general IS journals publish little GSS research (CAIS published none). Even more alarming is the fall in GSS share of JOC&EC over time. The European journals publish little GSS research perhaps indicating that GSS research is mostly a North American university phenomenon which is now fading, or perhaps GSS researchers are concentrating more effort on distributed collaboration technologies and their impact on virtual teams, for example.
Four issues stand out in Table 2. First, there much fewer NSS papers published (82) than GSS papers (383), so NSS is a substantially smaller field. Second, very few journals publish NSS research (only 6 of the 16 journals) so there are a limited number of outlets for NSS researchers. (Note that the other 10 journals in this study are not listed in Table 2 as they published no NSS research.) Third, the GD&N journal completely dominates NSS research (84.1% of total publications) reinforcing the limited publication outlets issue.
Finally, and more promisingly, NSS publication is growing (from 16 to 24 to 42 papers in the three eras) indicating a growing interest among researchers.
Table 2: NSS Papers by Journal
Journal 1990 - 1996 1997 - 2003 2004 - 2010 Total No. % of
Period No. % of
Period No. % of Period No. % of Sample
DSS 0 0.0 2 8.3 3 7.1 5 6.1
I&M 0 0.0 1 4.2 1 2.4 2 2.4
JMIS 1 6.3 1 4.2 1 2.4 3 3.7
JOC&EC 0 0.0 0 0.0 2 4.8 2 2.4 GD&N 15 93.8 20 83.3 34 81.0 69 84.1
MISQ 0 0.0 0 0.0 1 2.4 1 1.2
Total 16 100.0 24 100.0 42 100.0 82 100.0
Procedure
The protocol used to code each paper addressed each of the factors described in this paper. This protocol has been previously published in detail (Arnott and Pervan, 2005; Pervan et al., 2005; Pervan and Arnott, 2006) and is not provided here to save space.
Findings
The discussion above on publication rates and outlets addressed the first research question. An additional issue associated with publication was the declared support (if any) in terms of funding from competitive grants, industry, or the university itself.
Table 3a: GSS and NSS Financial Support
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
GSS Support 51 32.5 24 17.0 29 34.1 104 27.2
NSS Support 4 25.0 7 29.2 19 45.2 30 36.6
Table 3a reports the number and percentage of projects/papers supported financially and reveals that NSS research is more often supported in the most recent era in particular, and overall.
Table 3b below breaks down research funding by type (major competitive, industry, university) for both GSS and NSS. Where funded, both GSS and NSS researchers seem to be quite successful in winning major competitive grants, particularly more recently. However, neither is attracting funds from industry which is disappointing for applied disciplines such as GSS and NSS, and may reflect a low relevance in the eyes of potential industry research partners.
Table 3b: GSS and NSS Financial Support by Type
1990 - 1996 1997 - 2003 2004 - 2010 Total
GSS No. % No. % No. % No. %
Major competitive 30 58.9 19 79.2 26 89.5 75 72.1
Industry 22 43.1 2 8.4 2 6.8 26 25.0
University 14 27.4 8 33.4 9 30.9 31 29.8
51 24 29 104
NSS
Major competitive 3 75.0 6 85.7 17 89.4 26 86.7
Industry 0 0.0 0 0.0 1 5.3 1 3.3
University 2 50.0 1 14.3 3 15.8 6 20.0
4 7 19 30
In answering the second research question “What type and quality of research has been done?” we investigated the research epistemology, article type, strength of theoretical foundations and research methodology, and problem importance.
Table 4: GSS and NSS Research Epistemology
1990 - 1996 1997 - 2003 2004 - 2010 Total
GSS No. % No. % No. % No. %
Positivist 101 64.3 91 64.5 55 64.7 247 64.5
Interpretivist 7 4.5 17 12.1 6 7.1 30 7.8
N/A 49 31.2 33 23.4 24 28.2 106 27.7
157 141 85 383
NSS
Positivist 5 31.3 11 45.8 20 47.6 36 43.9
Interpretivist 0 0.0 1 4.2 0 0.0 1 1.2
N/A 11 68.8 12 50.0 22 52.4 45 54.9
16 24 42 82
Data on research epistemology appears in Table 4. In relation to GSS it appears that positivist research is dominant over all three eras (64.5% of papers) with only 7.8% of papers identified as interpretivist. In NSS research interpretivist research is virtually non-existent (1 paper out of 82!). Note that 27.7% of GSS papers and 54.9% of NSS papers had no identifiable epistemology. Chen and Hirschheim’s (2004) study of Information Systems research in general from 1991 to 2001 reported that 81% of papers had a positivist orientation with 19% using an interpretivist approach. These results confirm that, even more strongly than IS research in general, positivist research dominates both GSS and NSS.
There are a number of different approaches to classifying the type of research in addition to paradigm. The approach used in this project is a simplification of that used by Arnott and Pervan (2005). The different types are as listed in Tables 5a and 5b below.
Table 5a: GSS Article Type
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
Conceptual Study 33 21.0 24 17.0 9 10.6 66 17.2
Descriptive
Research 16 10.2 9 6.4 1 1.2 26 6.8
Experimental 45 28.7 55 39.0 27 31.8 127 33.2
Field Study 3 1.9 11 7.8 0 0.0 14 3.7
Case Study 19 12.1 17 12.1 12 14.1 48 12.5
Survey 7 4.5 1 0.7 5 5.9 13 3.4
Literature Review 2 1.3 5 3.5 1 1.2 8 2.1
Secondary Data 2 1.3 3 2.1 1 1.2 6 1.6
Action Research 0 0.0 2 1.4 3 3.5 5 1.3
Design Science 30 19.1 14 9.9 26 30.6 70 18.3
Total 157 141 85 383
Table 5a reveals that experimental research dominates GSS (33.2% of GSS papers) where university- developed GSS (GroupSystems, SAMM, Meetingworks) are studied in decision rooms on artificial problems with students as subjects. It is promising that design science studies are common, particularly more recently, suggesting that there may be new technologies and processes developed to support group work.
Table 5b: NSS Article Type
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
Conceptual Study 8 50.0 8 33.3 3 7.1 19 23.2
Descriptive
Research 2 12.5 5 20.8 0 0.0 7 8.5
Experimental 2 12.5 6 25.0 11 26.2 19 23.2
Field Study 0 0.0 0 0.0 0 0.0 0 0.0
Case Study 1 6.3 0 0.0 0 0.0 1 1.2
Survey 0 0.0 0 0.0 3 7.1 3 3.7
Literature Review 0 0.0 0 0.0 1 2.4 1 1.2
Secondary Data 0 0.0 0 0.0 2 4.8 2 2.4
Action Research 0 0.0 0 0.0 0 0.0 0 0.0
Design Science 3 18.8 5 20.8 22 52.4 30 36.6
Total 16 24 42 82
The major NSS article type is design science research (Hevner et al., 2004) reflecting an interest in developing new NSS technologies, processes, and methods. This was followed by conceptual studies and experiments
(both 23.2%). It should be noted that there are few field and case studies in NSS or GSS research. Clearly, despite the maturity of these research areas, there is still a need for more research involving real stakeholders tackling real problems using different types of system.
The quality of each paper was addressed by assessing the strength of its theoretical foundations and research methodology (strong, adequate, weak), as well as the importance of the problem (strategic, tactical, operational). Tables 6-8 contain these assessments for both GSS and NSS research.
Table 6: Strength of Theoretical Foundations
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
GSS
Strong 47 29.9 62 44.0 42 49.4 151 39.4
Adequate 60 38.2 56 39.7 35 41.2 151 39.4
Weak 50 31.8 23 16.3 8 9.4 81 21.1
Total 157 141 85 383
NSS
Strong 5 31.3 1 4.2 23 54.8 29 35.4
Adequate 7 43.8 14 58.3 15 35.7 36 43.9
Weak 4 25.0 9 37.5 4 9.5 17 20.7
Total 16 24 42 82
Overall both NSS and GSS papers could claim adequate theoretical foundations with around 20% weak for both GSS and NSS. However, an examination of the trend over time reveals that there has been some improvement in the 2004-2010 period with less than 10% in the weak category in both cases.
Another, deeper, assessment of the nature and quality of the theoretical foundations of GSS and NSS research was to carefully review the number of papers actually used in providing a theoretical foundation to the research. This was not a raw citation analysis, but was based on cited references actually being used to develop a theoretical model, form some hypotheses, or explain some results. Some 30.3% of GSS papers and 35.4% of NSS papers cited no ‘real’ references, 19.3% GSS and 23.2% NSS cited one, 15.1% GSS and 14.6% NSS cited two, and only 30.2% GSS and 26.8% NSS cited three or more. An examination of citations over time showed there was no significant improving trend. This demonstrates that much GSS and NSS research has limited theoretical foundation.
In GSS research the research methodologies are fairly weak and not improving over time, which occurs for a variety of reasons. These include poor choice of method, lack or reliability and validity of instruments, poor data collection choices, and so on. There is much room for improvement. Alarmingly, it is even worse in NSS research with only 8.5% strong and 63.4% weak. There has been a little improvement over time but with half of the papers in 2004-2010 weak on methodology there is cause for concern.
Table 7: Strength of Research Methodologies
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
GSS
Strong 36 22.9 49 34.8 27 31.8 112 29.2
Adequate 31 19.7 47 33.3 28 32.9 106 27.7
Weak 90 57.3 45 31.9 30 35.3 165 43.1
Total 157 141 85 383
NSS
Strong 0 0.0 1 4.2 6 14.3 7 8.5
Adequate 2 12.5 6 25.0 15 35.7 23 28.0
Weak 14 87.5 17 70.8 21 50.0 52 63.4
Total 16 24 42 82
Table 8: Importance of the Problem
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
GSS
Strategic 11 7.0 17 12.1 14 16.5 42 11.0
Tactical 28 17.8 31 22.0 17 20.0 76 19.8
Operational 118 75.2 93 66.0 54 63.5 265 69.2
Total 157 141 85 383
NSS
Strategic 1 6.3 2 8.3 7 16.7 10 12.2
Tactical 4 25.0 3 12.5 9 21.4 16 19.5
Operational 11 68.8 19 79.2 26 61.9 56 68.3
Total 16 24 42 82
Table 8 shows that most GSS research addresses operational problems with 69.2% at that level and that focus is fairly consistent over time. A similar finding is apparent for NSS research with 68.3% focused at operational level problems. This may relate to the emphasis on experimental research on artificial problems and a lack of interaction with managers concerned with more strategic issues.
In answering research question (c), “How relevant is GSS and NSS research?”, the practical relevance of the research in each article was analysed, both from an IS practitioner and a managerial user point of view.
Any professionally focused academic areas (like GSS and NSS) need a reasonable balance between theory development and application since research and practice inform each other (Galliers 1994). The assessment of practical relevance is a subjective judgement that was informed by the aims and objectives of the paper, the nature of the discussion, and in particular the content of the concluding comments of each paper. The
scale used for both types of relevance was high-medium-low and assessments erred on the generous side where possible.
Table 9: Relevance to IS Practitioners
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
GSS
High 3 1.9 6 4.3 1 1.2 10 2.6
Medium 34 21.7 25 17.7 18 21.2 77 20.1
Low 120 76.4 110 78.0 66 77.6 296 77.3
Total 157 141 85 383
NSS
High 0 0.0 1 4.2 0 0.0 1 1.2
Medium 3 18.8 4 16.7 3 7.1 10 12.2
Low 13 81.3 19 79.2 39 92.9 71 86.6
Total 16 24 42 82
Very few GSS papers were assessed as highly relevant to IS practitioners (only 2.6% overall) and most were assessed to be of low relevance. This is particularly in the light of the interest in design science (see Table 5a). The findings are even worse for NSS research with 1.2% of papers assessed as highly relevant to IS practitioners.
Table 10: Relevance to Managerial Users
1990 - 1996 1997 - 2003 2004 - 2010 Total
No. % No. % No. % No. %
GSS
High 10 6.4 12 8.5 12 14.1 34 8.9
Medium 44 28.0 37 26.2 33 38.8 114 29.9
Low 103 65.6 92 65.2 40 47.1 235 61.4
Total 157 141 85 383
NSS
High 0 0.0 3 12.5 2 4.8 5 6.1
Medium 3 18.8 3 12.5 13 31.0 19 23.2
Low 13 81.3 18 75.0 27 64.3 58 70.7
Total 16 24 42 82
The results for managerial relevance are slightly better than those for IS practitioner relevance for both NSS and GSS. However, the high relevance assessment occurred in only 8.9% of GSS papers and 6.1% of NSS papers and the vast majority of both GSS and NSS papers are in the low managerial relevance category.
While the project was initiated with a concern for the relevance of GSS and NSS research, we were surprised by the strength of this adverse finding. The relative lack of exposure of academics to contemporary professional practice is a particular problem.