Applying Large Language Models in Teaching Business English Writing: A Case Study of Business Proposal Writing

. With the maturing of artificial intelligence technology, the human-machine collaborative teaching model is gradually attracting attention. Exploration of the path and effectiveness of integrating Large Language Models (LLMs) into business writing teaching is urgently needed. This study takes the use of ERNIE Bot in teaching business proposal writing as an example, collecting and analyzing qualitative and quantitative data, and discussing the methods, effects and challenges of applying LLMs in teaching business English writing in universities in China. The results show that students using ERNIE Bot as an auxiliary tool demonstrate higher participation and enthusiasm in the proposal writing process, and their writing texts have significantly improved in terms of structural clarity and language accuracy. However, it has also been found that ERNIE Bot tends to recommend conventional expressions and structures, which to a certain extent limit students’ personalized expressions. Therefore, the leading role of the instructor should not be neglected.


Introduction
Amid the rapid evolution of artificial intelligence and machine learning technologies, Large Language Models (LLMs) in has been lucratively implemented in numerous natural language processing (NLP) functions including language production, textual error correction, and text summarization.In educational frameworks, the LLMs' adeptness in understanding and generating NLP can notably streamline learning processes and boost outcomes.This asserts a significant potential application in language writing instruction.
Mastering business writing is a core skill within the scope of business English.Consequently, finding efficient avenues to augment the quality of instruction has become a prevailing focus for education professionals within business language instruction.The task of crafting a business proposal, being constituent of intricate professional knowledge, specialized lexicon, and adherence to business etiquette hence poses a formidable challenge for students and individuals involved in business English learning and business communication.Alongside, with the surge in information technology, human-computer collaborative instructional models are drawing increasing acknowledgment.LLMs, akin to the language synthesis capabilities as seen in the Chat GPT series [1] and ERNIE Bot (a type of LLMs developed by Baidu China Co., Ltd., a leading Chinese multinational technology company), including aspects of grammar, vocabulary, context, and style, tremendously broadens its applicability in business writing instruction [2].Yet, research anchoring an effective implementation of LLMs in business English writing remains scant.
In light of the above, this study seeks to do an indepth exploration into the application of LLMs (specifically ERNIE Bot) in educational methodology, including the practical technique, pedagogical efficacy, as well as the intrinsic challenges.It could serve as reference in bolstering human-computer collaborative instruction in business language teaching.

Literature review
Owing to its commercial backdrop, business English writing, as compared to general English writing, boasts unique stylistic characteristics in terms of format, structure, and rhetorical strategies, making it an essential area of focus in English teaching [3].Business English writing not only requires speakers' English proficiency but also necessitates their thorough understanding of the business environment and specific business cultures.Although many courses and materials are specifically designed for business English writing, plenty of students still face challenges in accurately, fluently, and effectively using business English for writing purposes In recent years, there has been a notable intersection of information technology (IT) and language teaching, offering new perspectives on how educators approach this field.Various attempts to implement IT in teaching writing have been explored in China, with promising initiatives and results that enhance the learning experience significantly [4] .
Wang Haixiao (2014) explored some potential pedagogical strategies for English writing instruction reform in the era characterized by big data technology and proposed some coping measures for university English writing instructors [5].Chen Xiaoxiao and Ge Shili (2016) proposed that business English writing ability should be analyzed in the dynamic process divided into planning, writing, and completion stages [6].Chen Juanwen and Wang Na (2016), through empirical research, investigated the influence of digitized learning resources on enhancing students' English writing ability and shared that diversified, open, micro, and digitized learning resources could produce a positive and far-reaching impact on enhancing students' English writing ability [7].As to teaching method, Wang Wanrong (2019) contended that introducing case teaching approach in business English writing could provide students with a real-world business experience, thus facilitating the cultivation of applied business talents [8].Liang Haiying (2022) outlined the core idea of English writing teaching based on digital humanities based on their commonality with foreign language writing instruction, particularly discussing digital humanities tools deployable at various stages of process genre writing and their application methods [4].
Overall, an emerging trend of integrating (business) English writing with digital information technology has been observed in teaching and research, yet studies tracing the combination of the latest LLM technology such as with ChatGPT and ERNIE Bot for "humanmachine collaborative" teaching are urgently needed.

Research questions
Drawing upon the current state of business English writing instruction and research, the operational principles of LLMs and their present application status, the research questions are set as follows: (1) How do instructors utilize LLMs for designing and implementing business English writing teaching?
(2) What impact do LLMs have on students' business English writing abilities?
(3) What are the challenges and issues encountered when applying LLMs in business English writing teaching?

Subjects
The subjects of the study are from 2 natural classes of Business English course instructed by the first researcher of this paper.They are 78 third-year college students majoring in business from an applicationoriented college of business in Zhejiang, China.Among them there are 43 females and 35 males, possessing certain business knowledge and English ability (approximately at CSE Level 5 or IELTS 5.5).Before the teaching, their mastery of business English writing, especially complex business English texts, are not sufficient.Besides, only 7 of them have had experience with the LLMs.

Methodology
This study adopts a mixed-method approach, encapsulating both quantitative and qualitative research.Quantitative analysis allows the researchers to analyze numerical data through statistical methods, providing empirical evidence to answer research question two concerning the impact of LLMs on students' business English writing skills.Conversely, qualitative analysis aids in uncovering underlying reasons, motivations, and specific problems encountered when applying LLMshence addressing research question three.
Specifically, a case of teaching is reported.The case provides a detailed observational view of actual instances where ERNIE Bot is used in business English writing instruction.It constitutes a documented record of classroom interactions, students' performances, patterns of use, and practical challenges involving ERNIE Bot during the course of teaching.

Data collection and analysis
Data for the research was collected before, during, and after the case teaching, briefly listed as follows: Quantitative data: A five-scale questionnaire is developed (Cronbach α 0.921) and distributed pre-and post-case implementation.The questionnaire aids in capturing the possible change of perceptions, attitudes, or evaluations of students toward the use of LLMs in business English writing.
Qualitative data: Employing a content analysis approach, responses gathered from structured interviews of 6 students are evaluated.This aids in understanding deeper dynamics, including students' experiences, attitudes, perceived efficacy, and potential issues concerning the use of LLMs.

Design and implementation of the teaching case 4.1 Teaching background
The textbook used is "International Talent English Course (Intermediate Level)", conducting teaching based on workplace cases and tasks.Material arrangement of each unit in the textbook follows this sequence: Scenario Introduction -Task Description -Information Gathering -Skill Development -Task Practice -Vision Expansion.
The case was selected from the fourth unit, with the primary teaching theme of writing a business proposal ("Propose improvements to the sweeping robot").The teaching objectives of the case teaching are set as follows: (1) Knowledge objective: Help students understand and master the basic words, expressions, structure and writing skills for a business proposal; (2) Skill objective: Develop students' proficiency in effectively utilizing LLMs to assist in writing business proposals; (3) Value objective: Enhance students' digital literacy and higher-order thinking skills in using LLMs to discover and solve problems.

Teaching design and procedures
Before the class, the instructor, also the first researcher of the present study, introduced the students to LLMs and ways of intelligent questioning by referring to literature [9] [10].
In the class, the instructor first presented the scenario for a startup, tasked with drafting a compelling proposal.Then, it involved phases of Information Gathering followed by Skill Development and Task Practice sessions that encourage iterative improvement.Finally, Vision Expansion encouraged learners to apply these acquired skills to diverse business communication scenarios, fostering continuous improvement in their writing abilities.Throughout the teaching and learning, LLM empowerment was realized by using ERNIE Bot.

Phase 1: Scenario Introduction
Primary teaching activities: The instructor guided students to understand the real business situation of this proposal writing by watching a video.Students analyzed the keys to succeed in proposal writing tasks, and stimulated learning interest.The students also tried to draft a business proposal and selectively presented their proposals.
LLM empowerment: The instructor and students used ERNIE Bot to get more scenarios for writing business proposals, then discussed and shared some of the scenarios in class, enriching the case scenario corpus.
Prompts example: "Please provide some business cases required for writing a business proposal."

Phase 2: Task Description
Primary teaching activities: The instructor guided the students to refine and clearly define learning objectives and output tasks based on the given scenarios.
LLM empowerment: Students tried writing business proposals using ERNIE Bot.
Prompts example: "What materials are need to be prepared for writing a business proposal?"

Phase 3: Information Gathering
Primary teaching activities: Through a combination of teaching and training, the students worked in groups to choose a business topic, applied the points and methods of business proposal writing they just learned.LLM empowerment: Students used ERNIE Bot to review and improve their proposals; the prompt given should clearly state the evaluation criteria; groups used ERNIE Bot to provide feedback and discuss together.Prompts example: "As a senior instructor of a business English writing course, please check the structural completeness, accuracy of wording, and logical coherence of the following business proposal..."

Phase 4: Skill Construction
Primary teaching activities: Students first selected a business topic, then applied the essential writing tips and methods learned for writing business proposals, then they wrote their own draft of a business proposal, and optionally presented the draft proposal.
LLM Empowerment: Students used ERNIE Bot to check and improve their proposals, with prompts that clearly outlined evaluation criteria.They also used ERNIE Bot to provide feedback and discuss in groups.
Prompts example: "As a senior English instructor for business writing, please check the structural integrity, accurate wording, and logical coherence of the following business proposal..."

Phase 5: Task Practice
Primary teaching activities: Students combined the above content to write, edit, and evaluate the proposal.The instructor evaluated the proposal and provides suggestions and opinions.
LLM empowerment: Students used ERNIE Botto to write and edit proposals, including generating ideas, reorganizing and planning content, improving language expression, etc.; the instructor used ERNIE Botto to provide rewriting suggestions and ensure the evaluation was fair and objective.
Prompts example: "As a senior business English instructor, please provide 2-3 English business proposal templates.""Please evaluate the following business proposal and provide optimization suggestions from perspectives of structural completeness, language accuracy, and logic..."

Phase 6: Vision Expansion
Primary teaching activities: The instructor carried out supplementary teaching and broadened the topic, especially the cross-cultural factors in business English communication.
LLM empowerment: The instructor used ERNIE Bot to give further comments and modification suggestions on students' final written text, guiding students to expand their thinking.
Prompts example: "Please elaborate on the importance of business proposals in international business activities."

The Role of instructors in human-machine interaction
It was found that in such human-machine interaction, the instructor played a leading role in teaching design, classroom implementation, and teaching guidance.The role of the instructor has changed.Reflections from the instructor and peer discussions show that when utilizing ERNIE Bot in designing and implementing teaching, the instructor's role shifted from the traditional dispenser of knowledge to a guide and supervisor of learning.The instructor would focus more on the interactive dialogues between students and ERNIE Bot, providing targeted feedback and assessment and implementing personalized teaching based on students' needs.
Besides, the way of organizing classrooms has also changed.When ERNIE Bot was used in designing and implementing business English writing instruction in this study, it became an essential medium, making instructor-student interaction more diversified.

The Impact of applying LLMs on students
According to the data collected, the overall impact of applying ERNIE Bot in business English writing teaching on students was positive, mainly manifested in the following aspects.
(1) Improved writing ability.Students' writing samples shows that out of those students who employ ERNIE Bot to write business proposals, 80% significantly improved their writing efficiency and quality.Their proposal structures were clearer, arguments were more logical, and their language expressions are more accurate and professional.In selffeedback, 70% of students expressed that during interactions with ERNIE Bot, they gained new understandings and benefits regarding the use of English phrases, sentence structure, and logical argument development, shown in table 1.

Table 1. Descriptive data on the impact of applying LLM (ERNIE Bot) on students' writing
Note: For all the items, the Min. is 2.000, the Max. is 5.000.
Furthermore, a pearson correlation coefficient was used to express the strength of the relationship between the responses to "Do you know how to apply a LLM to business English writing?" and "Has the application of ERNIE Bot improved your efficiency in business English writing (such as writing business proposals)?" is 0.409, which is significant at the 0.01 level, indicating a significant positive correlation between the two, which means that those who knew more about how to LLMs became more efficient in business English writing, following in table 2. (2) Improved autonomous English learning ability.85% of the students believed that LLMs like ERNIE Bot can provide instant feedback and evaluation (shown in table 3), helping students promptly identify their mistakes and issues and make corresponding improvements.(3) Increased interest in learning.While using ERNIE Bot for business English writing, their learning experience was improved.Compared with traditional teaching, the interaction and immediacy of LLMs can enhance students' learning motivation and interest, making learning more exciting and fulfilling.83% of the students believed that using LLM-assisted writing gave them novel and stimulating learning experience, increasing their interest in studying, following in table 4. Furthermore, in the interview, they were asked: "How do you think the application of LLMs can better stimulate your interest or motivation in learning?"And some responded that they hoped that LLMs could help clarify their learning difficulties and provide them more authentic materials or cases.Table 4. Descriptive analysis of students' attitudes towards using LLMs Note: For both of the items, the Min. is 2.000, the Max. is 5.000.
Furthermore, results of the interview confirmed that ERNIE Bot was an interesting and critical tool for them to learn business English writing in this case.For example, one student mentioned that "I remember that when I couldn't recall how to start the conclusion part, I immediately thought of asking ERNIE Bot.It was interesting to see it generating the answer, and I at the same time got the expressions needed.However, in the past, facing this kind of problems, I would feel hesitant and just write down something perfunctorily"

Limitations of LLMs in business English writing teaching
There are also some limitations of applying LLMs in teaching business English writing in this study.As Floridi (2020) mentioned that GPT-3 also had some limitations [11].
In this study, for one thing, the students gave feedback in the interview that when using ERNIE Bot, it's hard to ensure the innovation and personalization of the suggested proposals because it tended to recommend conventional expressions and structures.Besides, 13% of them reflected that sometimes the suggestions from LLMs conflict with their initial intention and creativity.One of them recalled in the interview that "The suggested answer was too lengthy and covers too many points.And the ideas were too general."Consequently, they had to spend more time than expected on evaluating and selecting the suggestions given by ERNIE Bot.

Conclusion
This study exploits the application of LLMs, ERNIE Bot as an example, in the practice of teaching business English writing, emphasizing the importance of humanmachine collaboration.The major findings are as follows: (1) The advantages of the human-machine collaborative teaching model: First, it improves students' business English writing skills.Relying on LLMs, students can receive timely feedback on vocabulary, grammar, and text organization, as well as common expressions in business English and templates for drafting business proposals.This can enhance students' writing skills and language accuracy.Second, it enhances students' interest in writing.LLMs can provide a more efficient method of writing, which may boost students' interest in writing and improve their positive attitude towards writing tasks.
Third, it strengthens students' technological skills and attitude towards human-machine collaboration.Through writing practice using LLMs, students can gain a deeper understanding of the pros and cons of humanmachine collaboration, improving their technology skills.In addition, they become more willing to accept and use new technological tools for learning and work.
(2) The shortcomings and challenges of the humanmachine collaborative teaching model: LLMs like ERNIE Bot tend to recommend traditional and conventional expressions and structures, which may limit students' innovation and personalized expressions to some extent.Excessive reliance on those tools may cause students to lose control over basic writing skills, or have overly high expectations for artificial intelligence.
The results of this study may inspire more researchers or teachers to explore the application of LLMs in other foreign language teaching fields such as dialogue training and reading comprehension.Despite of the significantly positive impact of LLMs on teaching business writing, its limitations and issues should not be ignored.How instructors can properly and effectively guide students to use these tools still needs further research and discussions.

Table 3 .
Descriptive analysis of LLMs' influence on students' self learning Note: the Min. is 3.000, the Max. is 5.000.