This section presents the results of our interviews and miracle questions. The order in which we present the results is according to the order the miracle questions appeared in the survey. The order within the three sub-sections is chosen according to the flow of the narrative. For each sub-section, the most important key words are highlighted in bold & italics and visualised through a mind map per miracle question. These mind maps entail hyper-links to participant’s anecdotes and quotations relating to the respective keywords.
https://miro.com/app/board/o9J_lncuKIs=/?moveToWidget=3074457366642330977&cot=14
Miracle Questions: Mind Maps
This Miro Board consists of the 3 mind maps, visualising the most important key words mentioned by study participants in relation to the 3 miracle questions.
Navigation: One may extend the board to the screen and zoom in- and outwards. The links are clickable and lead to the respective anectodes and quotations.
The first miracle question investigates what output alternatives to a paper an astronomer would like to engage in. More than 300 study participants see no alternative to writing results up in the form of a paper, for mainly two reasons: First, many find it difficult to think of an alternative and second, many express it is simply the best way to distilling and disseminating their research. However, the sheer amount of alternative suggestions demonstrate that there is a desire for diversity in output formats. More specifically, study participants advocate for the freedom to adapt their output format to the present context. This sub-section presents a variety of suggested alternative forms of presenting one’s research than the current publication process.
First, study participants emphasise that the dissemination of knowledge is important to them. This includes reaching the community, but also outreach. Many therefore suggest a variety of easily digestible formats that shall be shared online and open access. Examples are online material/ lectures, online short articles, press releases, theatre style talks (e.g. TED talk), media appearances (news articles, TV, radio) and comic illustrations.
Second, many study participants advocate for improving the publication process. They criticise that non-English papers are not received by the astronomy community, putting those scientists at a disadvantage who cannot write proficient English papers and cannot afford appropriate proof-reading. Many address at the dysfunctional peer review system, where researchers often don’t have enough time next to their research to conduct it properly. Personal biases may also play a role since despite alleged anonymity of the author, quite often the peer reviewer can guess the paper’s origin. Some therefore advocate for open peer review to have transparency in the process. Furthermore, participants criticise that they have to do a bulk part of the editing workload despite the journal owning the rights to the paper. Some therefore advocate for the community owning the journal instead, so that researchers can set the rules of the game themselves.
Third, study participants advocate for a greater focus on quality than quantity. On the one hand this can be achieved by publishing less and potentially larger, more comprehensible papers. This would reduce the current information overload, where it is impossible to stay on top of the research and produce high quality research [IW9]. On the other hand, study participants also suggest to publish micro-publications (of partial results), which would make the dissemination of knowledge more agile. Micro-publications, however, entail the potential to increase information overload. Therefore, there is an ambivalent potential to increase research quality in both, micro-publications and macro-publications. Micro-publications have the potential to reach the community quicker and being more digestible, while macro-publications may decrease information overload and give a more complete overview of the entire path of the research process/ treatise, including failures along the way.
Fourth, negative results, as well as data and reduction code shall be publishable in some form and count as output. Data and code shall be stored in some sort of repository that can be linked with each other and the results. Software and data shall become more discoverable and citable and given equal value as papers in terms of output. Right now, the development of reduction code only counts towards one’s achievement if a whole paper can be dedicated to it. Furthermore, making data and pipeline code transparent would improve replicability and further advancement of knowledge, increasing research quality [e.g. IW4, IW5, IW6, IW7].
Fifth, related to a faster dissemination of knowledge and transparency, study participants value preprint repositories, such as ArXiv, as a valuable source of knowledge. They suggest to make ArXiv more interactive, where peers can comment and review the presented research. Furthermore, any documentation that reflects the research process, such as (lab) reports, manuals, research notes and notebooks, is perceived as a worthy output. Shared code notebooks, such as Jupyter/ IPython, shall be executable, interactive and linked to data.
Sixth, open discussions with peers and the community are not only valued as vital for the research process, but shall also be perceived as a form of output. This includes collaborations, seminars, meetings (e.g. with the research group), conferences, colloquia, giving talks, visits to institutions, giving workshops and short presentations. Knowledge shared in those discussions, shall be disseminated and freely shared online, such as in the form of recordings and conference proceedings.
Seventh, while some study participants point out that it’s important that an astronomer is involved in the whole research process, including writing up the results, some state that it’s not everyone’s talent to do so in a coherent and readable way. They wish for others (“ghostwriters”) who can communicate results better to write the paper. Language editors and graphic designers, who polish the papers, could assist this process.
Last, but not least, many study participants advocate for updating the antiquated paper style, which does not live up to the technical possibilities of the 21st century. The current linear style is perceived as boring to read. Moreover, in order to maintain readability, a linear paper may not contain too many details, such as derivations in calculations and theoretical background. This makes it difficult to follow the reasoning; astronomers in a different subfield may not at all understand the presented research, and astronomers in the same subfield may not be able to replicate exactly what was done. IW5, a former astronomer, now working for the journal of the American Astronomical Society, argues for “transitive referencing”. This is a citation system, which detects the (data) sources cited studies are built upon in order for those to also receive credit. The debate about style is closely related to the debate about whether a paper is the only way to communicate results. Many study participants advocate for a mixed style in order to present results more accessibly to colleagues and the public. Participants wish to enhance their publications not only with links to data, code and calculations used for the study, but also with visualisations and videos. Many media formats are suggested to enrich or (partly) substitute the paper: videos, books, podcasts, blog posts, social media and visualisations (e.g. plots, figures, charts, graphs, images, flow chart, VR experiences, art installations). Next to publications in multimedia formats, many study participants wish for the publication to be a “live format” (“live manner”, “living papers”, “dynamic” [MQ1]). This means that publications are updatable and include interaction and exchange with peers. A way to integrate multimedia and live formats is a knowledge repository in the shape of a Wiki. Study participants imagine this “encyclopaedia” to be continuously updated (including version control), hyperlinked, interactive, community-based reviewed, and available to everybody. This would include links memos, lab reports and other documentation as well as to data, code and calculations.
To summarise this sub-section, study participants find it important that knowledge gets “preserved” [IW10] in order to advance knowledge and so that nobody reinvents the wheel. While many participants think knowledge preservation in form of a paper is the only way, many others argue that they quite of lack in “digestibility”. Organising knowledge in a more innovative and interactive way than the current paper style would pay tribute to many alternative output formats mentioned before, such as including open discussions and increasing replicability. Moreover, formats like Wikis would give a better overview of the current state of knowledge, reducing information overload. This includes not having to paraphrase one’s own introduction time and time again when another paper is written based on the same study or theoretical background.
The second miracle question investigates what study participants wish with respect to what should count towards their performance assessment. This is related to the first miracle question insofar that output (currently in the form of peer-reviewed papers) is an important performance criterion. While the first miracle question resulted in a variety of different output formats that shall be recognised as part of an astronomer’s performance, this question goes beyond output and expands to other aspects of research.