Assessment Task IFN521 Foundations of DecisionScience

Assessment Task
IFN521 Foundations of Decision
Science
Semester 1 2023
Assessment 2 – Knowledge and
Name A presentation of theory related to a selected cognitive information process
Due End of Week 12 (with an essential component (Part A) presented to teaching staff in week 9, and formally submitted at the end of week)
9) (refer to Canvas for exact due dates)
Type Group-work based – Groups of 2
Weight 40% (indicative weighting)
Deliver Written document (Portable Document Format)
Submit PDF via Canvas
Skills Task
Rationale and Description
Evidence-based practice is an important aspect of Decision Science. In this assessment, you will create and conduct a hypothesis-based experiment involving the collection of data from human participants via crowdsourcing.
More specifically, this assessment involves researching human decision making by developing a crowdsourced research design, running the research design and writing a report based on the findings. This will demonstrate your ability to understand some of the relevant literature in the area, identify a method to learn more about how humans make decisions through experimentation, as well as collect and critically analyse human data.
You will use your knowledge and skills gained from the workshops and materials presented in the course, and apply them to a research problem. You will use your unique knowledge from the data you collect to provide a research report explaining what you have found, as well as suggest current or future technological innovations which may implement your findings.
Learning Outcomes
A successful completion of this task will demonstrate:
1. An ability to formulate a clear hypothesis associated with a given research topic, and justify the hypothesis by means of a concise analysis of relevant literature.
2. An ability to design an ethically sound experimental protocol to appropriately test the hypothesis.
3. An understanding of how experimentation can inform future innovations in decision support technologies.
4. An understanding of how humans make decisions while interacting with information.
5. An ability to analyse experimental data in order to determine whether the hypothesis is supported by the experimental data.
6. Effective written communication in the form of a research report which documents the experiment.
7. An understanding of requirements to conduct a study according to ethical guidelines.
8. An ability to work effectively in a group and manage equal contribution across team members
Essential Elements
You must submit a research report with the following sections:
1. Title
2. Abstract
3. Introduction
4. Method
5. Results
6. Discussion
7. References
8. Statement of Contribution
You are required to write the report as a team, and will be asked to provide evidence of this in the form of a statement of contribution. In writing your report, you must convince the reader that:
1. You have identified a justifiable hypothesis to address your research topic;
2. You have developed a feasible, practical methodology for testing the hypothesis, using contemporary tools and techniques and reasonable resources;
3. You understand the implications of your analysis of the data and effectively communicate your understanding in the context of the research problem; and
4. Your study adds value by identifying novel implications.
In order to convince the reader of these points your argumentation must be:
• Clear — Your writing must be easily understandable by a lay reader, avoiding uncommon terminology and abbreviations.
• Concise — You must express your ideas efficiently, so that key points are not obscured by irrelevant material.
• Coherent — Your arguments and the conclusions you draw must be structured logically.
• Convincing — The overall “story” you tell must be compelling and believable.
Information & links to resources to help you with writing research reports will be provided in the ‘detailed instructions’ section below.
Ethics
Please note that, as your research projects use human subjects, they are subject to ethical guidelines for human research. Research projects conducted in this unit are covered by ethics approval number xxxxxxxxxx. As such, the guidelines set by this approval must be followed, and any divergence from this will be considered as a breach. The teaching team’s approval of your methodology is therefore essential before commencing data collection. The teaching staff will ascertain whether your proposed methodology meets ethical requirements and will approve or reject it. If you commence any form of data collection from human subjects (whether on the prolific or any other platform, through social media, friends, etc.) without having your methodology approved by teaching staff, this will result in an automatic fail for this assignment.
Marking Criteria
This assessment is criteria referenced, meaning that your grade for the assessment will be given based on your ability to satisfy key criteria. This means that you need to ensure you are making your knowledge and understanding clear to the person marking your assignment with respect to each of the criteria. Study the attached ‘Criteria Sheet’ and to make sure you fully understand what is expected for each of the assessment criteria.
.
You will not receive marks or percentages for this assessment. Instead, you will receive an overall grade (e.g., pass - 4, high distinction - 7) based on the extent to which you meet the assessment criteria. The weighting of each criteria is provided in the criteria sheet.
Feedback
Essential step (contributing to Part A): You will supply a statement of your hypothesis, summary of your analysis of the literature to justify this hypothesis, and associated research design to test the hypothesis. These will be presented to the teaching staff during Week 9.
This will enable the teaching staff to provide formative feedback, enabling you to:
(a) understand where you can make improvements to ensure your experiment passes
Ethics Approval, and
(b) spread your effort more evenly through the duration of this assignment.
If the material you submit at the end of Week 9 does not pass Ethics Approval, you will be provided with data from an already conducted experiment to complete the assignment. In such a case, your final grade will be affected based on the criteria associated with experimental design and ethics. It is therefore highly recommended that you make use of opportunities for formative feedback as described above.
Groupwork
Note: You are required to perform this assignment in groups of two people. Groups of three or more are not permitted. Only in exceptional circumstances will students be allowed to individually attempt the assignment. You are responsible for forming groups starting from Week 7, and finalised by the beginning of the Week 8 tutorial.
Teaching staff will assist in facilitating the initial stages of group formation, but ultimately, forming groups is your responsibility. If you are struggling to form a group, please use Slack to find another student to form a group. If you have not formed a group prior to the tutorial in Week 8, and have not received permission to complete the assignment as an individual (in special circumstances), you will not receive approval to conduct the experiment, and this will affect grades in criteria associated with design and ethics (see rubric below).
Detailed Instructions
In order to help you achieve an excellent result in this assignment, please follow these steps: Part A - Weeks 7-9
Step 1 – Develop a preliminary hypothesis. In Week 7’s workshop, you will develop a preliminary hypothesis using a template.
Step 2 – Begin literature review: You will find and analyse literature to justify your hypothesis. Note that your analysis of the literature may require the hypothesis to be revised.
In order to focus your analysis of the literature to relevant sources, select 6-8 references that address the following question:
What evidence exists to justify why I would expect the hypothesis to be true?
Step 3 – Design experiment (with ethics documents): In Week 8’s tutorial, you will develop an appropriate experimental design to test your hypothesis including one or two Google forms including stimuli with questions which will be used to collect the experimental data.
This, along with completed ethics documents that will be made available on Canvas, will be presented in consultation by week 9 at the latest and will form part of your formative assessment.
Step 4 – Present hypothesis, justification, experiment, and ethics documents to the teaching staff for formative feedback: Create a presentation comprising 2 slides: The first slide contains a statement of your hypothesis using the correct format. The second slide contains3-4 dot points with references that summarize the justification of yout hypothesis.
Present your slides as well as your stimuli with questions (in Google Form(/s)) and Ethics documents to teaching staff in Week 9 (either during the live lecture or tutorial (A registration form will be provided to schedule your presentation). Note, you will have around 7 minutes for this, so you need to be prepared and succinct in your presentation in order to receive the most helpful feedback.
If you would like feedback on your experiment prior to Week 9, you can contact the teaching staff on Slack, or receive feedback during the tutorial in weeks 7 and 8. The latest date for receiving feedback from the teaching staff is the tutorial in Week 9.
Step 5 – Submit the experiment and ethics documents for final approval: Integrate the feedback supplied by appropriately editing your presentation, Google form(/s) or ethics documents, and submit via Canvas for final Ethics Approval by the end of Week 9 (see Canvas for exact due date). This is an essential element in order for the teaching staff to approve your experiment. Note that, this is the final chance to have your design approved. If it is not approved, you will not be able to conduct your experiment (per step 6). Instead,we will provide you with a pre-designed study with results that you can use to complete your assignment for steps 7 and 8. This will impact the grade awarded in the “Experimental Design and Ethics” criterion in your final submission.
Final submission for Part A DUE at the end of Week 9 (see Canvas for exact date)
Part B - Weeks 10-12
Step 6 – Upload your experiment onto the Prolific crowdsourcing platform: In order to do this, you will be given a username and password for a Prolific account with a certain amount of funds in order to pay participants from whom you will collect the data. Guides will be provided in order to upload your experiment and collect the data.
Step 7 – Analyse data: Once the data has been collected, you will analyse the data in order to determine if the data supports your hypothesis. Resources are provided to help you determine which analysis method to use.
Step 8 – Write research report: The final step involves writing a research report which documents your experiment. The required sections of the report are as follows.
Written Report – Essential Requirements:
1. Title. A concise, informative title for your experiment, which lists the group members in alphabetical order together with students numbers.
2. Abstract. A half-page summary of your entire research report. The summary concisely describes the following points:
• What you did
• Why you did it
• How you did it
• What you found
3. Introduction. This will be one to one and a half (1 – 1.5) pages long and describes the following points:
• A short background leading to why you would expect humans to make information-based decisions in a certain way, given your research topic
i. Use 6-8 relevant references/citations to justify your hypothesis*
• State the research topic and hypothesis of your experiment.
4. Methodology. This describes your experiment in enough detail for another researcher to replicate it, and includes:
• A summary of participants (e.g., describe who participants were, how many, etc.)
• Materials used (e.g., describe what the participants saw) • Experimental procedure (e.g., describe what the participants did)
5. Results. This describes the results of your experiment, and includes:
• Summary of data in table/graphical form
• Analysis method used and how it was applied to analyse the data
• Outline of the results from the data analysis
6. Discussion. This will be one to one and a half (1 – 1.5) pages long. This places your results back into the context you gave in your introduction, and demonstrates what your experiments has found, including:
• Whether the results confirm or disconfirm your hypothesis and what that implies for your research topic
• How these results relate to your previously referenced sources (from introduction)
• Limitations of your study
• Implications of your study for decision support technologies (e.g., search engines, recommendation systems, dashboards, etc.)
7. References. This is the list of all literature referenced in the body of your report. All references should be in APA format both in the body of the report and in the reference list.
8. Statement of Contribution. You will agree on your degree of contribution, which you will include in writing at the end of your document. This will be in the following format:
• Experimental design: Student 1 (%), Student 2 (%); Analysis: Student 1 (%),
Student 2 (%); Introduction: Student 1 (%) and Student 2 (%); Methodology:
Student 1 (%), Student 2 (%); Results: Student 1 (%), Student 2 (%); Discussion: Student 1 (%), Student 2 (%)
9. Appendix. In this section, provide the table(/s) of your raw results.
* Note that these references must be cited in the body of your introduction and listed in a reference list, rather than just being added as a bibliography/reading list at the end.
For information on how to write high quality reports, please see the following links (note, however, that we do not require you to write a complete introduction section, so please continue to refer back to the above for the essential requirements):
• https://www.citewrite.qut.edu.au/write/empiricalarticle.jsp
• http://www.discoveringstatistics.com/docs/writinglabreports.pdf
Clarification
If you require any clarification on the requirements of the assessment, as outlined in this document, please ensure you speak to teaching staff during workshop times or via Slack, as soon as possible to ensure you do not fall behind.
Please note, this clarification applies to assessment requirements as well as general questions about logistics involved in running crowdsourced studies. We cannot provide any specific feedback on your research or provide you with specific help which may provide you with an unfair advantage. In addition, if you need any help with writing, please see HiQ for assistance.
Resources
In addition to the links provided above and the preparatory materials and workshops included in this course, we will provide a “Crowdsource Guide”, which can be accessed on Canvas, and consists of a text-based and video-based guide to crowdsourcing. We will also provide a mandatory ethics training video that you will need to watch before being approved to begin collection of data from human participants. Please ensure you read and watch these resources before asking any questions about how to conduct crowdsourcing for this assignment.
Questions
Questions relating to the assessment should be directed initially to the teaching staff during the workshops and consultation times. The teaching staff may address these for the benefit of the whole class.
If you need to clarify something outside of these times, please direct your queries to the Slack team. A fellow student may have the same question and may benefit from the answer. Additionally, another student may know the answer to your question and be able to answer it at times when the teaching staff are unavailable (i.e. outside business hours). Teaching staff will monitor and respond to queries on the Slack channel, however, will only be able to do this during normal business hours.
Criteria Sheet – Assessment 2 Experimental Presentation of a Theory – IFN521 Foundations of Decision Science
7 – High Distinction 6 – Distinction 5 – Credit 4 – Pass 3 – Marginal Fail 2 - Fail 1 – Low Fail
Use of literature as supporting evidence (24% of total grade) – The ability to:
• draw on authoritative sources as needed to support a hypothesis and methodology (and demonstrate research capability by finding the majority of sources outside the unit set reading list)
• analyse, critique and evaluate literature sources
• understand the significance of the literature to the topic
7 6 5 4 3 2 1
• Citations are provided to support all significant points in the argument. • Hypothesis and methodology are very clearly and logically informed by the citations and argument provided • All sources are consistently relevant (to the research topic), current and authoritative.
• Referencing is thorough and consistently accurate. • Excellent level of critical analysis • Citations are provided to support almost all significant points in the argument.
• Hypothesis and methodology are clearly and logically informed by the citations and argument provided • Sources are mostly relevant (to the research topic), current and authoritative.
• One or two insignificant errors apparent in the referencing.
• High level of critical analysis
• Citations are provided to support all but one or two significant points in the argument.
• Hypothesis and methodology are mostly clearly and logically informed by the citations and argument provided • Sources are mostly relevant (to the research topic), current and authoritative.
• Referencing is slightly inconsistent or contains a few minor errors.
• Sound level of critical analysis • Citations are provided, but a few significant points in the argument lack supporting evidence.
• Hypothesis and methodology are somewhat clearly and logically informed by the citations and argument provided
• Half the sources are relevant (to the research topic), current and authoritative.
• Several noticeable referencing errors.
• Satisfactory level of critical analysis • Limited use of citations to support key parts of the argument.
• It is substantially unclear how hypothesis and methodology has been informed by the citations and argument provided
• A minority of sources are relevant (to the research topic) but may not be authoritative. • Some significant referencing errors. • Some evidence of
critical analysis • Inadequate citations to support the bulk of the argument.
• There is scant connection between the argument and citations used and the hypothesis and methodology • Sources are mostly irrelevant and/or not authoritative.
• Many significant referencing errors.
• Little evidence of critical analysis • No attempt to use the
literature to support argument
• The hypothesis and methodology do not appear to have been informed by the citations
and argument provided • Literature cited is irrelevant to the arguments.
• Referencing is absent or erratic.
• No evidence of critical analysis
Hypothesis (10% of total grade) – Ability to:
• Specify a well-formed hypothesis that is a precise, testable hypothesis appropriately expressed in terms of independent and dependent variables relevant to informationbased decisions
7 6 5 4 3 2 1
• Hypothesis is excellently formed, extremely relevant to information-based decisions and clearly testable via online crowdsourced experiment. • Hypothesis is very well formed, very relevant to information-based decisions and testable via online crowdsourced experiment.
• Hypothesis is well formed, relevant to information-based decisions and may be
testable via online crowdsourced experiment.
• Hypothesis is satisfactorily formed, reasonably relevant to information-based decisions, and/or may be difficult to test via online
crowdsourced experiment.
• Hypothesis is not well formed and lacks relevance to informationbased decisions, and/or may not be appropriate to be tested via online crowdsourced experiment.
• Hypothesis is unsatisfactory and largely irrelevant to informationbased decisions, and/or is not appropriate to be tested via online crowdsourced experiment.
• Hypothesis is either missing or totally unsatisfactory
Experimental Design and Ethics (15% of total grade) – Ability to:
• Design a feasible, stepwise research methodology to appropriately address the hypothesis in a way that would provide a reasonable contribution to knowledge
• Demonstrate understanding of Human Research Ethics guidelines
7 6 5 4 3 2 1
• It is made obvious that completing the process will meaningfully contribute to testing the
hypothesis, and reasonably contribute to knowledge on the subject
• Design adheres to
Human Research Ethics
Guidelines
• It is made very clear that completing the process will meaningfully contribute to testing the
hypothesis and reasonably contribute to knowledge on the subject
• Design adheres to
Human Research Ethics
Guidelines • It is made clear that completing the process will meaningfully contribute to testing the
hypothesis and reasonably contribute knowledge to the subject • Design adheres to
Human Research Ethics
Guidelines • It is generally clear that completing the process will meaningfully contribute to testing the
hypothesis and reasonably contribute knowledge to the subject • Design adheres to
Human Research Ethics
Guidelines • It is not entirely clear how completing the process will meaningfully contribute to testing the hypothesis, and/or is unlikely to provide a reasonable contribution to knowledge on the subject either in the way the design is constructed or executed, or due to the way the hypothesis is formed.
• Design does not adhere to Human
Research Ethics
Guidelines
• It not at all clear how completing the process will meaningfully contribute to testing the hypothesis, and/or is unlikely to provide a reasonable contribution to knowledge on the subject either in the way the design is constructed or executed, or due to the way the hypothesis is formed.
• Design does not adhere to Human
Research Ethics
Guidelines
• No clear attempt was submitted to design an experiment within the guidelines.
Continued on next page…
Research Methodology (8% of total grade) – Ability to:
• Describe a feasible, stepwise research methodology
• Identify the data collection and analysis steps needed to complete the process
• Estimate the resources needed to complete the project
7 6 5 4 3 2 1
• A comprehensive summary of participants is presented
• A comprehensive description of materials is presented
• The proposed research procedure is described extremely clearly, as a series of precise steps.
• It is made obvious that completing the process will contribute to testing the hypothesis.
• A mostly
comprehensive summary
of participants is presented • A mostly comprehensive
description of materials is presented
• The proposed research procedure is described very clearly, as a series of clear steps.
• It is made very clear that completing the process will contribute to testing the hypothesis.
• A clear summary of participants is presented, although lacks some information
• A clear description of materials is presented • The proposed research procedure is described clearly, as a series of generally well-described steps.
• It is made clear that completing the process will contribute to testing the hypothesis. • A summary of participants is presented, although lacks some clarity and information • A description of materials is presented, however, some clarity and information is lacking • The proposed research procedure is generally clear, but some steps need further explanation. • It is generally clear that completing the process will contribute to testing the hypothesis. • Participants involved in the study are not presented in enough
detail and/or with enough clarity
• Materials are not presented in enough
detail and/or with enough clarity
• The proposed research procedure is described weakly, or some key steps are unclear.
• It is not entirely clear how completing the process will contribute to testing the hypothesis. • Little detail is given about participants involved in the study • Little detail is given about materials used in study
• The proposed research procedure is described poorly and/or several key steps are described inadequately.
• It is not at all clear that completing the process will contribute to testing the hypothesis.
• No useful detail is given about participants
• No useful detail is given about materials used in study
• The proposed research
procedure is inadequately described.
• There is no convincing argument that completing the process will contribute to testing the hypothesis.
Analysis & Results (10% of total grade) – The ability to:
• Identify an appropriate analysis method
• Effectively execute the analysis method
• Appropriately present results
7 6 5 4 3 2 1
• Analysis method used
is appropriate for the
methodology
• Analysis is fully correct • Excellent presentation of results with complete detail
• Analysis method used is appropriate for the methodology, although a more appropriate method could have been used
• Analysis is largely correct
• Very good presentation of results with complete detail
• Analysis method used is mostly appropriate for
the methodology
• Analysis is basically correct
• Good presentation of results with some minor
details missing
• Analysis method used is somewhat appropriate for the methodology
• Analysis is satisfactory but with some non-trivial errors
• Adequate presentation of results with details
missing
• Analysis method used is applicable but not appropriate for the methodology
• Analysis has some significant errors • Barely satisfactory presentation of results with major detail missing
• Analysis method used is both inapplicable and inappropriate for the methodology
• Analysis is poorly executed
• Presentation of results is unsatisfactory
• No coherent analysis method is used
• No analysis is provided • Presentation of results
is incomprehensible
Discussion (28% of total grade) – The ability to:
• Relate the results to the research topic (RT) & hypothesis
• Discuss the results in the context of previously referenced studies, as appropriate
• Identify the limitations of the study
• Justify the implications of the study
7 6 5 4 3 2 1
• Discussion very clearly states how the results address the hypothesis and relate to the RT • Excellent critical discussion of results in the context of previously referenced studies
• Significant limitations of
the study are very clearly communicated
• Implications of the study are excellently communicated – creating a very clear and logical picture of the usefulness of the results, with a very high degree of novelty • Discussion clearly states how the results address the hypothesis and relate to the RT • Good critical discussion of results in the context of previously referenced studies
• Significant limitations of
the study are clearly communicated
• Implications of the study are very well communicated – creating a clear and logical picture of the usefulness of the results, with a high degree of novelty • Discussion creditably states how the results address the hypothesis and relate to the RT
• Well-written discussion of results in the context of previously referenced studies, however, little evidence of critical thought
• Significant limitations of the study are clearly communicated, although some obvious limitations are not addressed • Implications of the study are well communicated, and the usefulness of results is made clear, with a creditable degree of novelty
• Discussion satisfactorily states how the results address the hypothesis and relate to the RT
• Adequate discussion of results in the context of previously referenced studies, however, little evidence of critical thought
• Some limitations of the study are communicated with adequate clarity, although some obvious limitations are not addressed
• Implications of the study are adequately communicated, and the usefulness of results is made somewhat clear, with a passable degree of novelty
• Connection made between results and hypothesis and RT is limited
• Limited discussion of results in the context of previously referenced studies, with little to no evidence of critical thought
• Some limitations of the study are communicated with some clarity, although most obvious limitations are not addressed
• Implications of the study are somewhat communicated; however, their usefulness is lacking clarity and novelty
• Connection made between results and hypothesis RT is mostly unclear
• Limited discussion of results in the context of previously referenced studies, however it is not clear and with little to no evidence of critical thought • Attempted
communication of
limitations that are irrelevant and/or unclear • Implications of the study are somewhat
communicated; however, their usefulness is not made clear, and with no discernible novelty
• Results are not related to the hypothesis and RT • No discussion of results in the context of previously referenced studies is attempted
• No clear limitations are communicated
• Little to no clear or relevant implications of the study are presented.
Continued on next page…
Expression & Presentation (5% of total grade) – The ability to:
• use fluent language with correct grammar, spelling and punctuation
• use appropriate paragraph and sentence structures
• use appropriate style and tone of writing
• produce a professionally presented document
7 6 5 4 3 2 1
• Inclusion of all essential
components of the report
(Title, Abstract,
Introduction,
Methodology, Results, Discussion, References), with excellent document structure.
• Very clear yet concise writing throughout.
• Perfect use of standard grammar, spelling and punctuation.
• Polished professional appearance.
• Inclusion of all essential sections of the report
(Title, Abstract,
Introduction,
Methodology, Results Discussion, References), with very good document structure.
• Clear yet concise writing throughout.
• Grammar, spelling and punctuation mainly accurate.
• Professional presentation.
• Inclusion of all essential sections of the report
(Title, Abstract,
Introduction,
Methodology, Results, Discussion, References) with good document structure.
• Clear yet
concise writing in most parts.
• Grammar, spelling and punctuation creditably accurate.
• Neat and tidy presentation.
• One missing essential section of the report
(Title, Abstract,
Introduction,
Methodology, Results, Discussion, References), with generally good document structure. • Clear yet concise writing in general but with a few unclear passages. • Grammar and/or spelling and/or punctuation are satisfactory.
• Unprofessional, untidy, or unattractive presentation in a few places.
• Multiple missing essential sections of the report (Title, Abstract,
Introduction,
Methodology, Results, Discussion, References), and poor document structure.
• Several parts of the document are either too brief and unclear or contain significant amounts of irrelevant material.
• Grammar and/or spelling and/or punctuation contain significant errors.
• Unprofessional, untidy, or unattractive presentation in many places.
• Multiple missing essential sections of the report (Title, Abstract,
Introduction,
Methodology, Results, Discussion, References) and very poor document structure.
• Most parts of the document are either too brief and unclear or contain significant amounts of irrelevant material.
• Grammar, spelling and punctuation contain numerous and distracting errors.
• Unprofessional, untidy, or unattractive presentation in most places.
• Essential sections (Title, Abstract, Introduction,
Methodology, Results, Discussion, References) not present or not clearly outlined, and extremely poor document structure.
• Document is entirely unclear either because it is too brief and missing important points or consists of large amounts of irrelevant material.