Update on Overleaf.

article.Rtex

@@ -117,7 +117,7 @@ Despite this popularity, even highly accurate classifiers make errors that cause
 As we show in a systematic literature review of SML applications, 
 communication scholars largely ignore misclassification bias.
 In principle, existing statistical methods can use ``gold standard'' validation data, such as that created by human annotators, to correct misclassification bias and produce consistent estimates.
-We introduce and test such methods, including a new method we design and implement in the R package \texttt{misclassificationmodels}, via Monte-Carlo simulations designed to reveal each method's limitations. Based on our results, we recommend our method as it is versatile and efficient. In sum, automated classifiers, even those below common accuracy standards or making systematic misclassifications, can be useful for measurement with careful study design and appropriate error correction methods.
+We introduce and test such methods, including a new method we design and implement in the R package \texttt{misclassificationmodels}, via Monte-Carlo simulations designed to reveal each method's limitations, which we also release. Based on our results, we recommend our method as it is versatile and efficient. In sum, automated classifiers, even those below common accuracy standards or making systematic misclassifications, can be useful for measurement with careful study design and appropriate error correction methods.
 }
 
 % fix bug in apa7 package: https://tex.stackexchange.com/questions/645947/adding-appendices-in-toc-using-apa7-package
@@ -403,6 +403,10 @@ We now present four Monte Carlo simulations (\emph{Simulations 1a}, \emph{1b}, \
 
 Monte Carlo simulations are a tool for evaluating statistical methods, including (automated) content analysis \citep[e.g.,][]{song_validations_2020,bachl_correcting_2017,geis_statistical_2021, fong_machine_2021,zhang_how_2021}.
 They are defined by a data generating process from which datasets are repeatedly sampled. Repeating an analyses for each of these datasets provides an empirical distribution of results the analysis would obtain over study replications. Monte-carlo simulation affords exploration of finite-sample performance, robustness to assumption violations, comparison across several methods, and ease of interpretability \citep{mooney_monte_1997}. 
+Such simulations allow exploration of how results depend on assumptions about the data-generating process and analytical choices and are thus an important tool for designing studies that account for misclassification.
+Code for reproducing our simulations is available here: \url{https://osf.io/pyqf8/?view_only=c80e7b76d94645bd9543f04c2a95a87e}.}
+
+
 
 \subsection{Parameters of the Monte Carlo Simulations}
 
@@ -880,12 +884,12 @@ For more information about the package,  please see here: \url{https://osf.io/py
 
 \section{Additional Plots and Simulations}
 
-In addition to the results reported in the main paper, we include in the next section auxiliary plots from the main simulations.  Below, we present results from further simulations that show what happens when  the error model is mispeccified, how results vary with classifier predictivness or when the classified variable is not balanced, but skewed, and as the degree to which misclassification is systematic varies.  a 
+In addition to the results reported in the main paper, we include in the next section auxiliary plots from the main simulations.  Below, we present results from further simulations that show what happens when  the error model is misspecified, how results vary with classifier predictivness or when the classified variable is not balanced, but skewed, and as the degree to which misclassification is systematic varies.
 
 \subsection{Additional plots for Simulations 1 and 2}
 \label{appendix:main.sim.plots}
 
-\begin{figure}
+\begin{figure}[htbp!]
 <<example1.g,echo=FALSE, message=FALSE, warning=FALSE, result='asis', dev='pdf', fig.width=6, fig.asp=.65,cache=F>>=
 
 p <- plot.simulation.iv(plot.df.example.1,iv='z')
@@ -895,7 +899,7 @@ grid.draw(p)
 \caption{Estimates of $B_Z$ in \emph{simulation 1a}, multivariate regression with $X$ measured using machine learning and model accuracy independent of $X$, $Y$, and $Z$. All methods obtain precise and accurate estimates given sufficient validation data.}
 \end{figure}
 
-\begin{figure}
+\begin{figure}[htbp!]
 <<example2.g, echo=FALSE, message=FALSE, warning=FALSE, result='asis', dev='pdf', fig.width=6, fig.asp=.65,cache=F>>=
 p <- plot.simulation.iv(plot.df.example.2, iv='z')
 grid.draw(p)
@@ -903,7 +907,7 @@ grid.draw(p)
 \caption{Estimates of $B_Z$ in multivariate regression with $X$ measured using machine learning and model accuracy correlated with $X$ and $Y$ and error is differential.  Only multiple imputation and our MLA model with a full specification of the error model obtain consistent estimates of $B_X$.\label{fig:sim1b.z}}
 \end{figure}
 
-\begin{figure}
+\begin{figure}[htbp!]
 <<example3.z, echo=FALSE, message=FALSE, warning=FALSE, result='asis', dev='pdf', fig.width=6, fig.asp=.65,cache=F>>=
 #plot.df <- 
 p <- plot.simulation.dv(plot.df.example.3,'z')
@@ -912,7 +916,7 @@ grid.draw(p)
 \caption{Estimates of $B_Z$ in \emph{simulation 2a}, multivariate regression with $Y$ measured using an AC that makes errors. Only our MLA model with a full specification of the error model obtains consistent estimates.}
 \end{figure}
 
-\begin{figure}
+\begin{figure}[htbp!]
 <<example.4.x, echo=FALSE, message=FALSE, warning=FALSE, result='asis', dev='pdf', fig.width=6, fig.asp=.65,cache=F>>=
 #plot.df <- 
 p <- plot.simulation.dv(plot.df.example.4,'x')
@@ -921,6 +925,8 @@ grid.draw(p)
 \caption{Estimates of $B_X$ in \emph{simulation 2b} multivariate regression with $Y$ measured using machine learning, model accuracy correlated with $Z$ and $Y$ and differential error. Only our MLA model with a full specification of the error model obtains consistent estimates. \label{fig:sim2b.z}}
 \end{figure}
 
+\clearpage
+
 \subsection{Simulating what happens when an error model is misspecified.}
 \label{appendix:misspec}
 In simulations 1b and 2b, the MLA method was able to correct systematic misclassification using the error models in equations \ref{eq:covariate.reg.general} and \ref{eq:depvar.general}.
@@ -986,7 +992,7 @@ grid.draw(p)
 \label{fig:dv.noz}
 \end{figure}
 
-
+\clearpage
 
 \subsection{Simulating varying automatic classifier accuracy}
 
@@ -1021,14 +1027,12 @@ grid.draw(p)
 \label{fig:iv.predacc}
 \end{figure}
 
+\clearpage
 
 \subsection{Simulating misclassification in imbalanced variables}
 
 For simplicity, our main simulations have balanced classified variables.  But classifiers are often used to measure imbalanced variables, which can be more difficult to predict.  Here, we show that MLA correction performs similarly well with imbalanced classified variables.  Notably, the quality of uncertainty quantification of methods tends to degrade as imbalance increases by replicating versions of our simulations 1a and 2a having 5,000 classifications and 200 annotations.  This suggests that imbalanced data requires additional validation data for effective misclassification correction. 
 
-\subsubsection{Imbalance in classified independent variables}
-
-
 \begin{figure}[htpb!]
 \begin{subfigure}{0.95\textwidth}
 <<iv.imbalance.x, echo=FALSE, message=FALSE, warning=FALSE, result='asis', dev='pdf', fig.width=6, fig.asp=0.5,cache=F>>=
@@ -1077,6 +1081,8 @@ grid.draw(p)
 \label{fig:dv.predacc}
 \end{figure}
 
+\clearpage
+
 \subsection{Simulating a range of classifier biases}