Skip to main content
Footer-logo Scripted-horizontal-dark
Solutions Down-arrow-black
Book-closed
Content Intelligence
Custom analytics to map your content vs. industry and competitive trends and ensure your brand’s message meets the market.
Stars-02
Content Strategy
Daily ideas, recommendations and strategies to optimize performance across every marketing channel.
Life-buoy-01
Content Production
Create content at scale for every marketing channel in your brand’s unique voice.
Play-circle
Marketing Execution
Distribute the right content to the right customer no matter what platforms you use.
Industries Down-arrow-black
Agriculture
Art & Design
Automotive
Building Materials
Cannabis
Career
Construction
Counseling
Customer Service
Dental
Education
Energy & Environment
Engineering
Fashion & Beauty
Family Practice
Food & Beverage
Gaming
Health & Wellness
Healthcare
Higher Education
Home & Garden
Human Resources
Injury Law
Interior Design
IT & Security
Insurance
Legal
Manufacturing
Media & Entertainment
Medical Law
Nutrition
Parenting
Payments
Personal Finance
Real Estate
Relationships
Retail & Ecommerce
Religion & Spirituality
Restaurant and Bar
SaaS
Sales
Senior Services
Software
Sports & Fitness
Technology
Transportation & Logistics
Travel
Customers Down-arrow-black
Building-03
Enterprises
On-brand content that performs across all inbound, outbound and PR channels.
Share-07
Agencies
Workflow solutions that support content strategies and cost-effective creation for every client.
Tv-03
Media Publishers
Scalable content production that complements your existing editorial content.
File-06
Content Marketers
Insights, strategies and content to support your SEO, ABM, and social media marketing campaigns.
Volume-max
Brand Marketers
Subject Matter Experts that ensure your brand’s story is told with integrity and performance in mind.
Bar-chart-square-plus
Sales Teams
Everything you need to support 1:1 personalized outbound campaigns and get deals closed, quickly.
Resources Down-arrow-black
Book-open-01
Blog
Scripted has supported Content Marketers with insights for over a decade. Find everything you need here.
Lightbulb-05
Knowledge Base
Further resources on using Scripted and industry best practices as they evolve.
Speaker-02
Case Studies
Learn how Scripted customers have leveraged our platform to drive successful content marketing campaigns.
Briefcase-01
Writing Samples
See Scripted’s Subject Matter Expert writers and editors in action.
Backpack
Free Tools
Tons of AI-powered tools to make your job easier.
Pricing
Sign In GET STARTED
Menu button
x Close Menu Solutions Down-arrow-black
Book-closed
Content Intelligence
Custom analytics to map your content vs. industry and competitive trends and ensure your brand’s message meets the market.
Stars-02
Content Strategy
Daily ideas, recommendations and strategies to optimize performance across every marketing channel.
Life-buoy-01
Content Production
Create content at scale for every marketing channel in your brand’s unique voice.
Play-circle
Marketing Execution
Distribute the right content to the right customer no matter what platforms you use.
Industries Down-arrow-black
Agriculture
Art & Design
Automotive
Building Materials
Cannabis
Career
Construction
Counseling
Customer Service
Dental
Education
Energy & Environment
Engineering
Fashion & Beauty
Family Practice
Food & Beverage
Gaming
Health & Wellness
Healthcare
Higher Education
Home & Garden
Human Resources
Injury Law
Interior Design
IT & Security
Insurance
Legal
Manufacturing
Media & Entertainment
Medical Law
Nutrition
Parenting
Payments
Personal Finance
Real Estate
Relationships
Retail & Ecommerce
Religion & Spirituality
Restaurant and Bar
SaaS
Sales
Senior Services
Software
Sports & Fitness
Technology
Transportation & Logistics
Travel
Customers Down-arrow-black
Building-03
Enterprises
On-brand content that performs across all inbound, outbound and PR channels.
Share-07
Agencies
Workflow solutions that support content strategies and cost-effective creation for every client.
Tv-03
Media Publishers
Scalable content production that complements your existing editorial content.
File-06
Content Marketers
Insights, strategies and content to support your SEO, ABM, and social media marketing campaigns.
Volume-max
Brand Marketers
Subject Matter Experts that ensure your brand’s story is told with integrity and performance in mind.
Bar-chart-square-plus
Sales Teams
Everything you need to support 1:1 personalized outbound campaigns and get deals closed, quickly.
Resources Down-arrow-black
Book-open-01
Blog
Scripted has supported Content Marketers with insights for over a decade. Find everything you need here.
Lightbulb-05
Knowledge Base
Further resources on using Scripted and industry best practices as they evolve.
Speaker-02
Case Studies
Learn how Scripted customers have leveraged our platform to drive successful content marketing campaigns.
Briefcase-01
Writing Samples
See Scripted’s Subject Matter Expert writers and editors in action.
Backpack
Free Tools
Tons of AI-powered tools to make your job easier.
Plans Sign In Get Started
  1. Blog Home
  2. Scripted Writers
  3. A Text Is A Text Is A Text

A Text is a Text is a Text

I have a bachelor's degree in Literature, and at college I spent most of my time thinking, writing, and talking about texts. About what a text is says, what a text means, and what a text is. When I decided to make the switch from studying literature to Computer Science a year and a half ago, I would have never thought I would be fortunate enough to still be thinking about those very topics.

Here at Scripted, I work on automatically grouping similar texts or documents together---in Machine Learning terms, I'm talking about document clustering and classification. There are a variety of ways to accomplish this, and the 'best' algorithm for this goal isn't always a clear-cut choice, as it can be largely dependent on the reasons you're trying to group documents together.

One preprocessing requirement most of these algorithms have in common is the need to reduce a document to a 'bag of words'. That is, these algorithms aren't concerned with word order. They just look at how many times each word appears in a certain text, or we can take the bag of words model and then produce a sometimes more telling representation like tf*idf vectors. To get an idea of what this looks like, here are the top terms of this very blog post in bag of words form (term frequency or tf) and tf*idf form (term frequency multiplied by inverse document frequency) when considered against the entire Scripted corpus:

Term Frequency: tf*idf:











text
document
literature
computer
algorithm
grouping
vector
time
term
clustering
15
10
5
5
5
4
4
4
4
6
grouping
vector
literature
corpus
classifying
startling
arc
grouped
extracting
algorithm
0.7349142790721426
0.4409485674432856
0.2755928546520534
0.2204742837216428
0.1837285697680356
0.1377964273260267
0.1102371418608214
0.0918642848840178
0.0787408156148724
0.06263473969364851


Using representations like these can be incredibly effective for grouping documents together and, when clustering, they can be very useful for discovering some central topics the documents in a corpus are about. For example, here are few of the topics businesses have hired our writers to write about:










heart
blood
artery
pressure
high
vessel
beer
ale
brewing
flavor
hop
yeast
data
cloud
quality
information
storage
system
beach
resort
luxury
island
hotel
vacation


This type of grouping is useful to us as a kind of finger on the pulse of our writers, so that we can see who writes about what in more detail. But one thing about the bag of words model bothers me, perhaps irrationally so--it's a simplification that changes the very form of a document. Once converted, a text is no longer a text, but a vector. It's a simplification that's necessary for this type of grouping, but both the literature student and the computer scientist in me are saddened every time I convert a document to a vector. The literature student part of me is saddened because I know I am losing important features of the text like tone, style, narrative arc, and meaning. The computer scientist part of me is saddened for the exact same reason--because these features are ultimately information. Indeed, they are almost the entire reason we care about these documents at all.

This simplification is necessary, at least for the time being, because computers are still ineffective at extracting and handling these features. That is to say: computers are not good readers (and even worse writers). Advancements are always being made, but as of now it's necessary to break a text down into a bag of words so a clustering or classifying algorithm can process it. And I can't deny that there is something pleasantly surprising about the fact that the simplification of a text into a vector can be incredibly useful. My literature student's inclination is to resist anything but a holistic approach when analysing a text, but it's always startling to see how the hyper-specialized nature of an algorithm can produce fascinating results.

Published by Scripted Writers on Friday, July 20, 2012 in Featured, Staff.

Sign Up For Your 30 Day Free Trial Today!

You agree to Scripted’s Terms of Use and Privacy Policy.
Already have an account? Login
Member & Writer Services
Membership Plans Become A Writer Help Center Customer Login Writer Login
Resources
Hire Writers Scripted Blog Glossary Competitors Blog Ideas Products
Legal
Privacy Terms of Use Writer Services Agreement GDPR Trust
Follow Us





Scripted-horizontal-light
©2011-2025
Hubspot chat