Product Engineer Opportunity Map

05 · METHOD & LIMITS

How this was read, and what it can't tell you.

1. 01Qualitative researchWhat practitioners say a Product Engineer does, read closely from public practitioner sources and coded into themes.2026-09 → 10
2. 02Product Engineer Mental ModelThe working definition (v0.5): The Product Engineer Path in 4 steps and 11 practices.v0.5
3. 03CodebookOne rule per practice: what counts, what does not, and the boundary cases. Revised during this edition.v3
4. 04Collect postings77 public postings titled Product Engineer on LinkedIn, full text kept privately. A convenience sample.2026-09-24 → 10-07
5. 05First readClaude marks each practice a posting asks for, with a verbatim excerpt, strong or weak.2026-10-19
6. 06Blind second readsAnother reading of the same postings without seeing the first marks; agreement measured with Cohen's κ.2026-10-29 · 11-03
7. 07AdjudicationDisagreements decided against the codebook: 100 around The Product Engineer Path, 18 of 68 on it.2026-11-04
8. 08Opportunity MapFigures frozen and checked by script, then published here with excerpts, ratings and explanations.2026-11-05

About this research

This is an exploratory edition that also developed the instrument used to read the postings. Its framing is directed content analysis (Hsieh & Shannon, 2005): prior findings guide the categories used to analyse a text. Here, those categories come from my working definition of Product Engineer.

The approach provides a way to investigate; the codebook provides this study's rules for recognising each responsibility. The protocol specifies the sample, readers and checks. A methodological reference supports the approach, but does not validate our particular rules or the use of AI readers.

Testing and revising the coding rules is part of this approach (Assarroudi et al., 2018). In this edition, the same postings helped refine the rules and produced the results. Agreement measured before those revisions does not establish how consistently the final rules will work on new postings.

Reading the explanations

Each practice shows the excerpts retained for its current rating. A strong practice may also have weaker supporting excerpts; each excerpt keeps its own label. Recorded rationales are editorial paraphrases of reasons saved during adjudication. A recorded decision without a reason is identified as such. Reconstructed explanations are new interpretations by Codex, each reviewed by me before publication; they do not change the classifications or claim to recover the original reader's reasoning. Where no individual rationale is available, the report says so. The short rule summaries describe the current codebook and are not a new reading of every posting.

The sample

Postings found by searching the title on LinkedIn, collected from 2026-09-24 to 2026-10-07, with the full public text retrieved on 2026-10-07. Postings from Portugal were collected first and the others later through a different search, so this is a convenience sample, not a random one; the map treats all 77 as one set: the country chart shows where the postings came from, and there are no comparisons between countries. Four postings are short summary ads: where they say nothing, I read "can't tell", not "not asked for". Links may have closed since.

The readings

First read (2026-10-19). Claude coded every posting against the 4 steps of The Product Engineer Path and 11 practices, with a verbatim excerpt for each mark. The counts on this page come from that reading after adjudication: reviewing disagreements and deciding which marks to keep.

The Product Engineer Path: blind second read (2026-11-03). Claude Opus reread all 77 postings across the 8 practices within The Product Engineer Path, without seeing the first reader's marks. This replaces the partial check of 2026-10-20, whose raw output was lost. The replacement's raw results are retained.

The readings used the same model family, Claude, so they may share errors. Cohen's κ measures agreement beyond that expected from the readers' marking rates; it does not establish that either interpretation is correct. These values compare the independent readings before adjudication.

The Product Engineer Path: agreement on each step

Step	Same reading	Cohen's κ
Problem	73 / 77	0.84
Decide	68 / 77	0.70
Build	76 / 77	0.00*
Learn	64 / 77	0.66

*For Build, the second reader marked every posting present, while the first marked all but one. With this pattern, expected agreement equals observed agreement and κ is zero despite high agreement, a known κ paradox (Feinstein & Cicchetti, 1990). It is not a useful standalone summary here. These are step-level results; practice-level results are separate in the data.

Decisions on The Product Engineer Path. Of 68 disagreements at practice level, I decided 18: the cases initially flagged for closer review and the weak cases that could change coverage of The Product Engineer Path. The remaining 50 weak marks appeared only in the second reading. They remain absent from the current counts and are included only in the sensitivity scenario above.

The practices around The Product Engineer Path (2026-10-29). A blind second read by Codex covered Keep the system healthy, Work across disciplines and Room to work in all 77 postings. Agreement was low: Cohen's κ 0.13, 0.22 and 0.10, respectively. Review found missed requests in the first read and overly broad interpretations in the second.

Decisions around The Product Engineer Path. Each of the 100 disagreements was classified against a revised codebook; I decided the borderline ones, reading the full text. I then checked a random sample of 15 of the 57 marks added without my decision; all 15 held. This was a spot-check, not an independent human reading of the full sample.

Data

Everything on this page is drawn from one file: data.json (JSON, edition sep-oct-2026, postings collected 2026-09-24 to 2026-10-07). You can check every count here against it.

• jobs: one entry per posting, with company, title, public link, country, work arrangement, Tech and our own summary. steps gives each step as strong, weak or absent; practices gives each practice's excerpts (q, verbatim, 25 words or fewer) and their evidence (s).
• distribution, asked, countries, regimes: the counts behind the figures above.
• agreement: Cohen's κ and disagreements per practice and step, and the verification sample.
• assessments: the coding rules and the decisions taken on disagreements.

Not included: the full posting text, which belongs to the employers and the job board, and the readers' working notes. Work arrangement and Tech come from one reader, with no second reading.

Reuse. The summaries, marks, counts and decisions are licensed under CC BY 4.0: reuse them with credit to Adilson Chicoria, Product Engineer Opportunity Map, edition Sep–Oct 2026, theproductengineer.net. The verbatim excerpts remain the employers' words, quoted with a link to each posting; the licence does not cover them.

References

These support the approach and the measures used, not this map's specific results or the use of AI readers.

1. Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9). doi:10.1177/1049732305276687. The directed approach: categories from prior research.
2. Assarroudi, A., Heshmati Nabavi, F., Armat, M. R., Ebadi, A., & Vaismoradi, M. (2018). Directed qualitative content analysis: the description and elaboration of its underpinning methods and data analysis process. Journal of Research in Nursing, 23(1). PMC7932246. Preparing, testing and revising the codebook.
3. O'Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: debates and practical guidelines. International Journal of Qualitative Methods, 19. doi:10.1177/1609406919899220. Independent coding, agreement and resolving disagreements.
4. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1). doi:10.1177/001316446002000104. Cohen's κ.
5. McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica, 22(3). PMC3900052. Reading κ alongside raw agreement.
6. Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43(6). doi:10.1016/0895-4356(90)90158-L. Why κ is zero for Build despite 76 of 77 matching.
7. Thabane, L., et al. (2013). A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Medical Research Methodology, 13. doi:10.1186/1471-2288-13-92. Sensitivity scenarios, as distinct from confidence intervals.
8. Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1). doi:10.11648/j.ajtas.20160501.11. What a convenience sample can and cannot show.

Limits and the next edition

• The codebook and protocol changed during this edition; readers also changed with availability. These results describe this sample and helped develop the instrument. They do not establish how common these responsibilities are across the market.
• The low agreement on the practices around The Product Engineer Path makes their counts particularly tentative. Resolving disagreements produces the reported counts; it does not improve the agreement measured between the original readings.
• The readers are AI models. I made the human decisions and also write this series; I was not an independent blind reader.
• Postings show what employers ask for in writing. They do not establish what the person actually does or how much autonomy they have.
• The next edition is planned with a protocol fixed before collection, readers from different model families, and an independent blind human reading of a subsample. It will test the refined rules on new postings.
• The full posting text belongs to the employers and the job board and is not shown here. The page uses short verbatim excerpts and summaries in my own words.