Distribution experiment · Beginner

Comment-First: An Experiment in Distributing AI Content

Comment-first: эксперимент с распространением AI-материалов
Temporary fallback cover; replace in editorial pass.

Do comments in other people’s discussions generate more replies than publishing your own material? Here is a repeatable way to find out without pretending that a small sample proves a universal rule.

20-minute read Published November 8, 2026 Updated November 8, 2026

The question is simple: if you have limited time, should you write a new post or add a useful comment to an existing discussion? The answer is not “comments always win” or “publishing is better.” The answer depends on what you measure, how you match the two activities, and whether your sample is large enough to support the conclusion.

This article defines a small, honest experiment that a beginner can repeat. It compares two distribution paths:

The protocol records replies, reach, impressions when available, time spent, and delayed responses. It also records what the experiment cannot establish.

The concrete case

Imagine that you are preparing a short explanation of an AI workflow. You have two possible uses for one writing session:

  1. Find an active discussion where people are already asking a related question, then write a focused response that solves one part of the problem.
  2. Publish your own explanation as a new post and wait for people to discover it.

Both actions produce AI content if the subject is AI, but they enter different contexts. A comment joins an existing conversation with an existing audience. A new post must create its own initial context. That difference is the reason a comparison is useful—and also the reason the comparison must be designed carefully.

Suppose the practical goal is conversation rather than vanity metrics. Define the primary outcome before starting:

“Within seven days, how many direct replies does each distribution path generate per hour of work?”

This definition prevents a common mistake: declaring a winner because one item received more views when the actual goal was replies.

What to measure

Use one primary metric and a short set of secondary metrics. A small experiment becomes difficult to interpret when every visible number is treated as equally important.

Primary metric: direct replies

Count replies that directly respond to the comment or published item. Decide in advance whether you will include:

A practical rule is to count human replies that are visible to the author and connected to the item. Record the rule in your notes and apply it to every observation.

Secondary metrics

Reach
The number of unique people shown the item, if the platform reports it. If only impressions are available, label the field “impressions” rather than calling it reach.
Impressions
The number of times the item was displayed. One person may create multiple impressions.
Reply rate
Replies divided by reach or impressions. State which denominator you used.
Time to first reply
The elapsed time between publication and the first qualifying reply.
Conversation depth
The number of reply levels or the length of the reply chain. Define the counting method before collecting data.
Work time
The minutes spent finding a discussion, drafting, revising, publishing, and responding during the measurement window.

When a platform does not expose a metric, enter “not available.” Do not replace a missing value with zero. Zero means the measured result was zero; “not available” means the platform did not provide evidence.

The minimum viable protocol

For a first run, use a paired design with at least four comment-first items and four publish-first items. More observations improve stability, but this protocol does not prescribe a magic sample size. The result should be described as directional until repeated.

Step 1: Write the question and stopping rule

Write these lines before publishing anything:

Question:
Which path generates more direct replies per hour: comment-first or publish-first?

Observation window:
Seven days after each item is published.

Primary metric:
Qualifying human replies per hour of work.

Stopping rule:
Four items per path, or the agreed larger sample, with no substitutions
after seeing the results.

The stopping rule matters because it prevents p-hacking-like behavior: changing the sample or outcome after seeing an attractive result. You do not need to use that technical label in your report, but you do need to avoid the behavior.

Step 2: Define eligibility

Choose the platforms and topics in advance. For example:

For a comment-first item, the discussion should be active enough that a reply could reasonably be seen. Record its age, visible activity, and approximate audience context. For a publish-first item, record the planned format, topic, and time of publication.

Step 3: Match the work

Do not compare a two-minute comment with a two-hour article and then attribute every difference to distribution path. Set a time budget. One simple choice is 30 minutes per item, including drafting and editing. If an item takes longer, record the actual time and report the difference.

Match the content as closely as practical:

Matching does not make the comparison a perfect controlled experiment. It only reduces obvious differences.

Step 4: Use a neutral content brief

Prepare one brief for each topic so the writing task is comparable:

Topic:
Audience:
Problem to answer:
One concrete example:
One limitation:
Call to action:
Maximum length:
Time budget:

For a comment, adapt the brief to the discussion instead of pasting a generic promotional message. For a published item, make the same useful point in a standalone form.

Step 5: Record the baseline

Before posting, record the fields that can affect visibility:

item_id:
path: comment-first | publish-first
platform:
topic:
date_time_published:
discussion_age_minutes:  # comment-first only
visible_prior_replies:
follower_or_audience_context:
word_count:
planned_time_budget_minutes:

Do not collect names, email addresses, private messages, or other personal data. Use an internal item ID rather than copying identifying information into the dataset.

Step 6: Publish and respond consistently

Publish during the preselected schedule. After publication, use the same response rule for both paths. For example, you may respond to genuine follow-up questions within the first 24 hours, but not start unrelated outreach.

Record when you respond. Your own follow-up can influence the number of replies, so it is part of the treatment—not an invisible adjustment.

Step 7: Check at fixed times

Check each item at the same intervals, such as:

At each check, record the visible counts. If the platform changes a count later, keep the earlier observation and add a later observation rather than overwriting the history.

Step 8: Close the observation window

At seven days, calculate the primary metric using the same formula for every item:

reply_rate_per_hour = qualifying_replies / actual_work_hours

If you want a reach-normalized secondary metric, define it explicitly:

replies_per_1,000_reached = qualifying_replies / reach * 1,000

Do not calculate a reach-normalized rate when reach is unavailable. Do not infer reach from likes or follower count.

A compact data sheet

A spreadsheet is sufficient. These columns are enough for a first run:

item_id,path,platform,topic,published_at,closed_at,
actual_minutes,word_count,qualifying_replies,
reach,impressions,time_to_first_reply_minutes,
follow_up_replies,notes

Example rows below are fictional and show the format only. They are not measurements from Agent Lab Journal or from any named platform.

Illustrative dataset structure
item_id path actual_minutes qualifying_replies reach
C-01 comment-first 28 Example value Example value
P-01 publish-first 31 Example value Example value

Replace “Example value” with observed data only after the item has been checked. If a field is unavailable, use “not available.”

Analysis without overclaiming

Start with item-level results, not a single grand total. For each path, report:

The median is useful because one unusually successful item can distort an average. Still, with only a few observations, the median is not magically reliable. Show the individual values and the range.

A simple comparison table can look like this:

Report template
Measure Comment-first Publish-first
Items observed Observed count Observed count
Median replies per item Observed value Observed value
Median replies per hour Observed value Observed value
Median reach Observed value or unavailable Observed value or unavailable
Observed range Lowest–highest Lowest–highest

Use cautious language:

Avoid statements such as “comments are better for growth” unless you have a much larger, well-controlled body of evidence and a clearly defined meaning of “better.”

Verification checklist

Before writing the conclusion, verify the record rather than relying on memory:

  1. Every item has a path, publication time, and item ID.
  2. The primary metric was defined before the results were reviewed.
  3. The same reply-counting rule was applied to both paths.
  4. Work time includes drafting, editing, publishing, and allowed follow-up.
  5. Reach, impressions, and replies are not treated as interchangeable.
  6. Unavailable values are marked as unavailable, not zero.
  7. All observations use the same closing window.
  8. Late replies are recorded consistently.
  9. Examples are labeled as examples.
  10. No private or identifying information is included.

If you use a spreadsheet, add a second person or a later self-review to check formulas. A minimal formula audit can be performed by recalculating a few rows by hand:

actual_hours = actual_minutes / 60
replies_per_hour = qualifying_replies / actual_hours

# Example calculation structure only:
# actual_minutes = 30
# qualifying_replies = observed_count
# replies_per_hour = observed_count / 0.5

The example intentionally leaves the count undefined. Insert a measured value only after collecting it.

Failure cases and what they mean

The discussion disappears

A moderator may remove the discussion, the author may restrict it, or the platform may change visibility. Mark the item as interrupted. Do not silently replace it with a new item after seeing how the other path performed. Report the missing observation and decide whether to restart the whole paired run.

The platform gives different analytics

One item may expose reach while another exposes only impressions. Preserve both fields and make the mismatch explicit. You can still compare direct replies per hour, but you cannot make a clean reach-rate comparison for unavailable values.

One item goes unusually viral

Keep it in the dataset if it followed the protocol. Add a sensitivity view that reports the results with and without that item, clearly labeled. Removing an inconvenient result without a predeclared rule creates a biased analysis.

The comment is answered by the discussion author

Do not automatically exclude it. The author’s reply is part of the visible outcome if it satisfies your predeclared definition. You may add a secondary field for “reply from discussion author” so the reader can inspect the composition of the result.

The published item receives no reach

That is an observation, not a reason to invent an estimate. Check whether the item was actually published and visible, then record the platform’s available evidence. If distribution was interrupted by a technical error, classify the item as invalid under the protocol and explain why.

You change the writing style halfway through

Record the change. A stronger hook, different length, or new format may be the cause of a later difference. If the change is substantial, treat the run as exploratory and repeat with a stable brief.

Small-sample limitations

Four items per path can reveal operational problems and generate a useful next question. It cannot establish a general law about comments, publishing, AI content, or a platform’s algorithm.

Small samples are vulnerable to:

There is also a selection bias risk. You choose which discussions are worth commenting on, so the comment-first path may benefit from your ability to identify promising conversations. That is a real part of the workflow, but it means the result measures the whole strategy—finding plus commenting—not just the act of typing a comment.

Likewise, a published item may have a longer useful life than the seven-day window captures. A fixed window makes comparison possible, but it may undercount delayed discovery. If delayed distribution matters, add a 30-day follow-up for every item.

Improving the next run

After the first run, choose one improvement rather than changing everything at once:

If the goal is deciding how to spend tomorrow’s writing time, replies per hour may be the most actionable measure. If the goal is long-term discovery, reach over 30 days and qualified follow-up may matter more. The best metric is the one connected to the decision you will make.

What a responsible conclusion looks like

A final report can be short:

“We compared [number] comment-first items with [number] publish-first items over [window]. The primary outcome was qualifying replies per hour. The observed median was [value] for comment-first and [value] for publish-first. Reach was available for [scope]. Because the sample was small and the items were not randomly assigned, this result is directional. The next test will [specific change].”

This format keeps the evidence close to the claim. It tells the reader what happened, what was measured, and what remains uncertain.

Final takeaway

Comment-first distribution is worth testing because it starts inside an existing conversation. Publishing first is worth testing because a standalone item may have a different discovery path and a longer shelf life. Neither path should be declared superior from intuition, a single successful post, or a small collection of screenshots.

Define the outcome, match the work, record the baseline, check at fixed times, preserve missing data, and report the limits. That is enough to turn “I feel like comments work better” into a repeatable question with inspectable evidence.

For related procedures, see the guides and the glossary.

We publish what works for us—and implement the same solutions for your business. We design AI automation, Telegram bots, chats, and AI agents for real-world processes. Discuss your project →