arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Mochen Yang

16 papers · 2018–2026

2018: 12019: 12020: 22021: 02022: 22023: 32024: 32025: 22026: 2
2018peak 3 in 20232026

Frequent co-authors

Papers

showing 1–16 of 16 RSS

Understanding User-Generated Content and Customer Engagement on Facebook Business Pages

Mochen Yang, Yuqing Ren, Gediminas Adomavičius

Sociology and Political Science · Information Systems Research · · cited 252× · DOI ↗ · more like this ·

With the growth and prevalence of social media platforms, many companies have been using them to engage with customers and encourage user-generated content about their products and services. In this paper, we analyze user-generated posts from the Facebook business pages of multiple companies across several industries to understand what users post on Facebook business pages and how post valence and content characteristics affect engagement, measured as the number of likes and comments received by a post. Our analysis demonstrates that negative posts are significantly more prevalent than positive posts, and negative posts also tend to attract more likes and more comments than positive posts. Importantly, engagement depends not only on the valence of a post but also on the specific post content. We observe three types of customer complaints respectively related to product and service…

Mind the Gap: Accounting for Measurement Error and Misclassification in Variables Generated via Data Mining

Mochen Yang, Gediminas Adomavičius, Gordon Burtch +1 more

Sociology and Political Science · Information Systems Research · · cited 95× · DOI ↗ · more like this ·

The application of predictive data mining techniques in information systems research has grown in recent years, likely because of their effectiveness and scalability in extracting information from large amounts of data. A number of scholars have sought to combine data mining with traditional econometric analyses. Typically, data mining methods are first used to generate new variables (e.g., text sentiment), which are added into subsequent econometric models as independent regressors. However, because prediction is almost always imperfect, variables generated from the first-stage data mining models inevitably contain measurement error or misclassification. These errors, if ignored, can introduce systematic biases into the second-stage econometric estimations and threaten the validity of statistical inference. In this commentary, we examine the nature of this bias, both analytically and…

<i>When Algorithms Err</i> : Differential Impact of Early vs. Late Errors on Users’ Reliance on Algorithms

Antino Kim, Mochen Yang, Jingjing Zhang

Computer Science Applications · ACM Transactions on Computer-Human Interaction · · cited 27× · DOI ↗ · more like this ·

Errors are a natural part of predictive algorithms, but may discourage users from relying on algorithms. We conduct two experiments to demonstrate that reliance on a predictive algorithm following a substantial error is affected by (i) when the error occurs and (ii) how the algorithm is used in the decision-making process. We find that the impact of an error on reliance depends on whether the error occurs early (i.e., when users first start using the algorithm) or late (i.e., after users have used the algorithm for an extended period). While an early error results in substantial and persistent reliance reduction, a late error affects reliance only temporarily and to a lesser extent. However, when users have more control over how to use the algorithm’s predictions, error timing ceases to have a significant impact. Our work advances the understanding of algorithm aversion and informs the…

Integrating Behavioral, Economic, and Technical Insights to Understand and Address Algorithmic Bias: A Human-Centric Perspective

Gediminas Adomavičius, Mochen Yang

Safety Research · ACM Transactions on Management Information Systems · · cited 24× · DOI ↗ · more like this ·

Many important decisions are increasingly being made with the help of information systems that use artificial intelligence and machine learning models. These computational models are designed to discover useful patterns from large amounts of data, which augment human capabilities to make decisions in various application domains. However, there are growing concerns regarding the ethics challenges faced by these automated decision-making (ADM) models, most notably on the issue of algorithmic bias , in which the models systematically produce less favorable (i.e., unfair) decisions for certain groups of people. In this commentary, we argue that algorithmic bias is not just a technical (e.g., computational or statistical) problem, and its successful resolution requires deep insights into individual and organizational behavior, economic incentives, as well as complex dynamics of the…

Understanding Partnership Formation and Repeated Contributions in Federated Learning: An Analytical Investigation

Xuan Bi, Alok Gupta, Mochen Yang

Information Systems · Management Science · · cited 22× · DOI ↗ · more like this ·

Limited access to large-scale data is a key obstacle to building machine learning (ML) applications in practice, partly due to a reluctance of information exchange among data owners out of privacy and data security concerns. To address this “information silo” problem, federated learning (FL) techniques have been proposed to enable decentralized model training via an orchestrating central server and have received increasing attention in several industries (including healthcare and finance). Despite its superior privacy protection property, adoption of FL is limited by a lack of systematic understanding of its underlying economics. In this paper, we take an analytical approach to answer two questions: (1) when do data owners prefer to form a FL partnership over building ML models by themselves and (2) how can different contractual mechanisms be used to promote repeated contributions to FL…

Engaging Users on Social Media Business Pages: The Roles of User Comments and Firm Responses

Xiaoye Cheng, Hillol Bala, Mochen Yang

Sociology and Political Science · MIS Quarterly · · cited 13× · DOI ↗ · more like this ·

Firms must strategically manage their responses to user comments to keep users engaged on their social media business pages. The question of whether, how, and when a firm should respond to user comments to achieve favorable outcomes is of great interest to researchers and practitioners. We focus on these questions and study the effects of initial user comments and firm responses on subsequent user engagement on social media business pages. In particular, we theorize and examine how two features of initial user comments (i.e., sentiment and controversialness) and two features of firm responses (i.e., uniqueness and timeliness) jointly affect the volume and sentiment of subsequent user comments. By analyzing data from the Facebook business pages of multiple U.S. retailers (10,312 firm posts from 37 firms and over 1 million user comments), we found that firms are more likely to respond to…

Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

Mochen Yang, Edward McFowland III, Gordon Burtch +1 more

econ.EM · · cited 8× · arXiv ↗ · more like this ·

Combining machine learning with econometric analysis is becoming increasingly prevalent in both research and practice. A common empirical strategy involves the application of predictive modeling techniques to 'mine' variables of interest from available data, followed by the inclusion of those variables into an econometric framework, with the objective of estimating causal effects. Recent work highlights that, because the predictions from machine learning models are inevitably imperfect, econometric analyses based on the predicted variables are likely to suffer from bias due to measurement error. We propose a novel approach to mitigate these biases, leveraging the ensemble learning technique known as the random forest. We propose employing random forest not just for prediction, but also for generating instrumental variables to address the measurement error embedded in the prediction. The…

Engagement by Design

Mochen Yang, Yuqing Ren, Gediminas Adomavičius

Sociology and Political Science · ACM Transactions on Computer-Human Interaction · · cited 7× · DOI ↗ · more like this ·

We study the impact and interplay of social design features on the engagement behaviors toward user-generated content on Facebook business pages. By examining the introduction of the “Reactions” feature on Facebook, we aim to understand how the introduction of a new engagement feature affects the overall engagement activities and the use of existing engagement features. We found evidence of a positive effect of Reactions on overall engagement levels. Furthermore, the introduction of the Reactions feature had heterogeneous effects on the use of existing engagement features. Posts that received Reactions also ended up receiving more Likes and Comments than what they would have received before the feature change. However, the opposite is true for posts that received no Reactions, although the effect sizes were small. These effects were detected within the first four weeks after the feature…

What, Why, and How: An Empiricist’s Guide to Double/Debiased Machine Learning

Bowen Shi, Xiaojie Mao, Mochen Yang +1 more

Artificial Intelligence · Information Systems Research · · cited 5× · DOI ↗ · more like this ·

We provide an introduction to double/debiased machine learning (DML), a framework that enables effect estimation when dealing with complex, high-dimensional data. In many empirical analyses, especially in fields such as information systems, researchers face difficult choices about which control variables to include and how to model their relationships with the outcome. These modeling decisions can significantly change results, leading to uncertainty about which findings are reliable. DML offers a practical solution by combining modern machine learning with rigorous statistical inference. The idea is to let flexible ML models (such as random forests or gradient boosting) capture complex relationships among control variables while still delivering reliable estimates for the key effect of interest. DML can be applied to many familiar research designs, including standard regression with…

Regurgitative Training: The Value of Real Data in Training Large Language Models

Jinghui Zhang, Dandan Qiao, Mochen Yang +1 more

cs.CL · · cited 3× · arXiv ↗ · more like this ·

What happens if we train a new Large Language Model (LLM) using data that are at least partially generated by other LLMs? The explosive success of LLMs means that a substantial amount of content online will be generated by LLMs rather than humans, which will inevitably enter the training datasets of next-generation LLMs. We evaluate the implications of such "regurgitative training" on LLM performance. Through fine-tuning GPT-3.5 with data generated either by itself or by other LLMs in a machine translation task, we find strong evidence that regurgitative training clearly handicaps the performance of LLMs. The same performance loss of regurgitative training is observed on transformer models that we train from scratch. We find suggestive evidence that the performance disadvantage of regurgitative training can be attributed to at least two mechanisms: (1) higher error rates and (2) lower…

EnsembleIV: Creating Instrumental Variables from Ensemble Learners for Robust Statistical Inference with ML-Generated Variables

Gordon Burtch, Edward McFowland III, Mochen Yang +1 more

econ.EM · · cited 2× · arXiv ↗ · more like this ·

Despite increasing popularity in empirical studies, the integration of machine learning generated variables into regression models for statistical inference suffers from the measurement error problem, which can bias estimation and threaten the validity of inferences. In this paper, we develop a novel approach to alleviate associated estimation biases. Our proposed approach, EnsembleIV, creates valid and strong instrumental variables from weak learners in an ensemble model, and uses them to obtain consistent estimates that are robust against the measurement error problem. Our empirical evaluations, using both synthetic and real-world datasets, show that EnsembleIV can effectively reduce estimation biases across several common regression specifications, and can be combined with modern deep learning techniques when dealing with unstructured data.

Welfare and Fairness Dynamics in Federated Learning: A Client Selection Perspective

Yash Travadi, Le Peng, Xuan Bi +2 more

cs.LG · · cited 2× · arXiv ↗ · more like this ·

Federated learning (FL) is a privacy-preserving learning technique that enables distributed computing devices to train shared learning models across data silos collaboratively. Existing FL works mostly focus on designing advanced FL algorithms to improve the model performance. However, the economic considerations of the clients, such as fairness and incentive, are yet to be fully explored. Without such considerations, self-motivated clients may lose interest and leave the federation. To address this problem, we designed a novel incentive mechanism that involves a client selection process to remove low-quality clients and a money transfer process to ensure a fair reward distribution. Our experimental results strongly demonstrate that the proposed incentive mechanism can effectively improve the duration and fairness of the federation.

Robustness is important: Limitations of LLMs for predictions on tabular data

Hejia Liu, Mochen Yang, Gediminas Adomavičius

General Social Sciences · PNAS Nexus · · cited 0× · DOI ↗ · more like this ·

Large language models (LLMs) are being applied in a wide array of settings, well beyond typical language-oriented use cases. In particular, LLMs are increasingly used as a plug-and-play method for generating predictions on tabular data. Prior work has shown that LLMs, via in-context learning (ICL) or supervised fine-tuning (SFT), perform comparably with many tabular supervised learning techniques. However, we identify a critical vulnerability of using LLMs for tabular prediction-making changes to data representation that are completely irrelevant to the underlying learning task can drastically alter LLMs' predictions on the same data. For example, simply changing variable names can sway the size of prediction error by as much as 82% in certain settings. Such prediction sensitivity with respect to task-irrelevant variations manifests under both ICL and SFT, for both close-weight and…

Cost-Aware Calibration of Classifiers

Mochen Yang, Xuan Bi

Artificial Intelligence · INFORMS Journal on Data Science · · cited 0× · DOI ↗ · more like this ·

Most classification techniques in machine learning are able to produce probability predictions in addition to class predictions. However, these predicted probabilities are often not well calibrated in that they deviate from the actual outcome rates (i.e., the proportion of data instances that actually belong to a certain class). A lack of calibration can jeopardize downstream decision tasks that rely on accurate probability predictions. Although several post hoc calibration methods have been proposed, they generally do not consider the potentially asymmetric costs associated with overprediction versus underprediction. In this research, we formally define the problem of cost-aware calibration and propose a metric to quantify the cost of miscalibration for a given classifier. Next, we propose three approaches to achieve cost-aware calibration, two of which are cost-aware adaptations of…

Large Language Models Can Explain Experts Better Than Experts Themselves

Mina Cho, Russell J. Funk, Alok Gupta +1 more

cs.HC · · arXiv ↗ · more like this ·

Capturing tacit knowledge, or the "know-how" embedded in individual experiences and contexts, is a central challenge that limits the preservation and transfer of valuable expertise. Traditional methods rely on experts' ability to articulate their own reasoning, even though much of tacit knowledge is difficult to verbalize. Here, we show that Large Language Models (LLMs) can uncover tacit knowledge from expert behavior and support knowledge transfer from experts to novices. We first identify prompting strategies that improve LLMs' ability to externalize tacit knowledge (i.e., transform tacit into explicit knowledge), and then evaluate the downstream utility of the resulting LLM-externalized tacit knowledge. Our analysis reveals that LLM-externalized tacit knowledge incorporating expert behavior improves decision quality and helps reduce the expert-novice score gap, outperforming…

Robustness is Important: Limitations of LLMs for Data Fitting

Hejia Liu, Mochen Yang, Gediminas Adomavicius

cs.LG · · arXiv ↗ · more like this ·

Large Language Models (LLMs) are being applied in a wide array of settings, well beyond the typical language-oriented use cases. In particular, LLMs are increasingly used as a plug-and-play method for fitting data and generating predictions. Prior work has shown that LLMs, via in-context learning or supervised fine-tuning, can perform competitively with many tabular supervised learning techniques in terms of predictive performance. However, we identify a critical vulnerability of using LLMs for data fitting -- making changes to data representation that are completely irrelevant to the underlying learning task can drastically alter LLMs' predictions on the same data. For example, simply changing variable names can sway the size of prediction error by as much as 82% in certain settings. Such prediction sensitivity with respect to task-irrelevant variations manifests under both in-context…