Open work
The one place we track unfinished work across the data pipeline. Not tied to a fiscal year or a migration — items get added and closed, the file stays.
Every item has the same four parts, and an item missing any of them is a bug in this doc:
- Unfinished — what is not done, concretely enough to act on
- How we know — the evidence, with the date it was checked
- Blocked on — what has to happen first, and who owns that
- Where — the file, survey, board or object involved
An item with no date-checked is untrustworthy: re-verify it or delete it. When you close an item, delete it — do not leave a tombstone. If closing it taught you something durable, that belongs in DECISIONS.md, which is where reasoning lives so it survives the item being closed.
This file was previously
FY27_MIGRATION.md, whose numbered sections started at 5 because 1–4 had been deleted as they completed — taking the reasoning with them. Headings here are descriptive rather than numbered for that reason.
Live deploy status does not belong in this file. It goes stale within days. ARCHITECTURE.md is auto-generated from the Posit Connect API precisely so that nobody hand-maintains a table of what is deployed and what is failing. Check there.
FY26-tab readers awaiting FY27 tabs
Unfinished. Seven files still read FY26 … tabs from the old Master Survey Input sheet (11jlo9UeWxZ…) and cannot move to the FY27 sheet (1hbs5d1uf2xq…) until equivalent FY27 tabs exist. Moving them now would hard- error. Separately, update_coaching_roster_27.qmd writes to FY26 Teacher Roster by Site, which has no FY27 equivalent on either sheet.
How we know. Checked 2026-08-06 (commit 9ed7cccc): the new sheet has no FY26 Official Site List, FY26 School/District Selection for Sites, or FY26 Teacher Roster by Site tab. Not re-verified since — re-check before acting.
Blocked on. L&R creating the FY27 tabs on the new sheet.
Where.
| File | Tab it needs |
|---|---|
posit_automations/automated_slide_decks.qmd |
FY26 Official Site List |
posit_automations/cap_targets/R/functions.R |
FY26 School/District |
posit_automations/coaching_action_plans/debug_cap_sheet.qmd |
FY26 School/District |
posit_automations/coaching_action_plans/create_cap_folders_in_google_drive.qmd |
FY26 School/District + FY26 Site List |
dashboards/posit-connect/cap_naming_validator/app.R |
FY26 School/District |
api/tl-survey-comm-api/plumber.R |
FY26 School/District |
Two of these are live services rather than batch jobs — the cap_naming_validator dashboard and the tl-survey-comm-api plumber endpoint — so they will serve FY26 data to callers who expect FY27 until they move.
update_deck_list.qmd was on this list and moved to the FY27 sheet on 2026-08-07 — see update_deck_list is running on placeholder inputs.
Reasoning for the two-sheet split: DECISIONS.md.
Whether FY27 needs its own Coach-Teacher Recorded survey
Unfinished. update_coach_selection.qmd and update_coachee_selection_27.qmd both target the SY25-26 Coach-Teacher Recorded Conversations survey while their other targets are FY27. This is deliberate, not an oversight, but it is unresolved: either FY27 genuinely reuses the SY25-26 survey, or a FY27 copy needs creating and the ids bumping in both files.
How we know. Checked 2026-08-07 against the Qualtrics API across all 190 surveys in the account. No survey matches FY27 / SY26-27 Coach-Teacher Recorded. The newest is SV_b9pikOzhbXvOapw “Coach-Teacher Recorded Conversations (SY25-26)”, last modified 2026-08-07 — so it is in active use.
Blocked on. L&R deciding whether FY27 wants a separate survey. Do not create one speculatively; a second live survey splits the response data.
Where. ctr_id in posit_automations/data_updates/update_coach_selection.qmd; the ctrc row of survey_targets in update_coachee_selection_27.qmd.
Deprecate tlShiny::subsites in the tlShiny repo
Unfinished. tlShiny::subsites is superseded by the derived registry at tl-surveys-27/subsite_question_registry_fy27.parquet, but the object still exists and is still exported from the tlShiny package. It needs a formal deprecation there.
There is a live bug to fix in the same pass: data-raw/setup_data.R contains two overlapping subsites <- … blocks, both calling use_data(overwrite = TRUE), so only the second ever takes effect. Anyone editing the first block sees no change and has no idea why.
How we know. Checked 2026-08-06 (commit 95fc347e). The same check found the object agreed with the live Educator Survey on only 35 of 38 mappings.
Blocked on. Nothing external — this is our work, in a separate repo.
Where. ../tlShiny/data-raw/setup_data.R (separate repo). Last consumer in this repo is posit_automations/tracker_27.qmd, which reads the registry and falls back to tlShiny::subsites with a warning.
Reasoning: DECISIONS.md.
FY27 site-question response backfill
Unfinished. When the six FY27 surveys were split off the shared site-selection reference survey, responses recorded before the split became unreachable from the export — site reads as NA for every pre-split FY27 response and no old-QID column appears. This was being backfilled by hand. Whether it finished is unrecorded.
How we know. Documented 2026-08-04; never verified as complete. Qualtrics has no API to edit recorded responses, so the fix is manual and there is no programmatic check. Confirm the current state before assuming either way.
Blocked on. Someone confirming the hand backfill completed, or finishing it.
Where. The six FY27 surveys that own a local site block — Educator Survey, IPG, Student Work, Mid/End of Year Lead, Enabling Conditions, Teaching Lab Student.
A related live consequence, already fixed but worth knowing: scripts calling data_setup(api = FALSE) have no __js_site_api fallback, so their site stays NA. update_contact_lead.R split rows on str_detect(site, "ny_"), which is NA for a missing site, and both branches dropped those rows — the script silently wrote nothing. Fixed to treat NA as non-NY.
Educator survey pre/post cutoffs are a zero-row template
Unfinished. tl-monday/ed_survey_cutoff_26_27.parquet exists only as a zero-row template carrying the FY26 schema (name, Internal Project Name, Content Area, Educator Survey, Classroom Observations, Student Work, ed_survey_cutoff). It needs populating from the FY27 partnership board, the way ed_survey_cutoff_25_26.parquet is produced.
Until then every FY27 response is classified by the standard split (prepost_split_date <- as.Date("2027-04-01")). That is correct for most sites and wrong for any site with a non-standard post window — the FY26 equivalent had 44 rows. This is an approximation, not a failure.
Zero rows is indistinguishable from “no site needs an override”. Do not treat the object’s existence as evidence the cutoffs are configured.
How we know. Checked 2026-08-04. The script runs and reads zero overrides.
Blocked on. FY27 partnerships being loaded onto the partnership board (L&R) — the same board that blocks tracker_27 and update_deck_list below.
Where. tl-monday/ed_survey_cutoff_26_27.parquet, read by update_educator_survey.R (ec2_tl_scripts), which uses only name and ed_survey_cutoff.
FY27 Teaching Lab Student Survey is inactive
Unfinished. SV_cDg05ZZPWuH94km exists but is not collecting — the only FY27 survey still inactive. Five steps to finish:
- Activate
SV_cDg05ZZPWuH94kmin Qualtrics. - Confirm the response schema matches what
data_setup(race_ss = TRUE)expects. The FY27 educator survey dropped three questions, so assume drift until checked. - In
update_student_survey.R: swap the survey id, shift the window to2026-07-01/2027-07-01, change the bucket totl-surveys-27. - Freeze the FY26 version as
update_student_survey_26.R(precedent:update_facilitation_competency_25.R). - Add the web-service task on the FY27 survey pointing at
/update/update_survey_s3?survey=student_survey.
How we know. Checked 2026-08-04. Downstream: update_student_survey.R is deliberately left on the FY26 survey (SV_6A29Ff4ZhgcbtZA) writing to tl-surveys-26, which is harmless; TeachingLab::get_student_survey(year = "26_27") resolves to tl-surveys-27/student_survey.parquet, which does not exist, so any caller passing 26_27 errors; tracker_27 references this survey.
Blocked on. L&R activating the survey. Steps 2–5 are ours once it is live.
Where. SV_cDg05ZZPWuH94km; update_student_survey.R (ec2_tl_scripts).
No FY27 Facilitation Competency survey
Unfinished. No FY27 equivalent exists; FY26 is SV_8ldVeywLkOMgugK. Once a FY27 copy is created, apply the same five steps as the Student Survey item above.
How we know. Checked 2026-08-04.
Blocked on. L&R creating the FY27 copy.
Where. update_facilitation_competency.R (ec2_tl_scripts) stays on FY26 → tl-surveys-26. posit_automations/validation/qualtrics_checks.qmd keeps fac_comp_id <- "SV_8ldVeywLkOMgugK" with a comment — it is the one non-FY27 id left in that file.
update_deck_list is running on placeholder inputs
Unfinished. The job has been pointed at FY27, but two of its four inputs are stand-ins and must be replaced before FY27 is real.
| Input | State |
|---|---|
| FY27 Official Site List | Real — 1hbs5d1uf2xq…, 44 sites |
deck_folder |
Real — 1BbtlcIdWbw8-00K5jH98l5AfQTT_z3gm “FY27 Automated Decks” |
| Partnerships board | PLACEHOLDER — no FY27 board exists |
| Template deck |
PLACEHOLDER — reusing 1JOX8Dg5… “1- AUTOMATIONS TEMPLATE FY26 Baseline” |
Because there is no FY27 Partnerships Information & Tracking board on Monday, partnership_info is a fake tibble built from the FY27 Official Site List: every site, with content_area_sites (NY_D9/D11/D12/D13/D75) split into Math and ELA rows the way the board would carry them. Consequence: it requests a deck for every FY27 site with no filtering on the board’s readiness columns — 49 decks. The real board is what normally narrows that. The placeholder block in [slide-decks-needed] carries the exact code to restore, including the (Master Survey Input) rename-back.
Two further notes. The Drive folder was created by the service account so it currently sits in Shared with me rather than beside “FY26 Automated Decks” — move it; the folder id does not change when you do. And [make-slides] trashes and recreates every deck in deck_folder on each daily run, so the FY26 decks are no longer being maintained.
readr must stay declared. range_speedread() needs it at runtime but only Suggests it, so the deploy scanner does not see it. It used to arrive transitively via TeachingLab::get_monday_board(); the placeholder board removed that call and the next deploy failed with “The package readr is required to use range_speedread()”. The setup chunk now carries an explicit requireNamespace("readr") — do not delete it, including when restoring the real board fetch.
How we know. Checked 2026-08-07. Deployed the same day, job rNVM726ivxS8JrAL exit 0, 49 decks in the FY27 folder.
Blocked on. L&R creating the FY27 Partnerships Information & Tracking board. That same board also unblocks tracker_27’s scaffold guard — do both together.
Where. posit_automations/data_updates/update_deck_list.qmd; Monday board 9397372037 is the FY26 original.
NY_CUNY/UA — fixed 2026-08-07. It previously never got a deck: named_needed rewrote it to NY_CUNY-UA, then the %in% which_slides_still_needed filter compared against the slashed form and dropped it (49 sites in, 48 decks out). NY_CUNY-UA is now canonical and the hyphen is applied on both sides of the filter. The FY26 code still carries the original bug if anyone revives it.
Qualtrics workflows on the FY27 surveys are unverified
Unfinished. The ec2 fetchers only run when Qualtrics POSTs to /update/update_survey_s3?survey=<name>. Nobody has confirmed the web-service tasks were configured on the FY27 surveys rather than only the FY26 ones. If they were not, the FY27 fetchers never fire and tl-surveys-27 only updates on a manual run_all_files.r.
How we know. Checked 2026-08-04 and could not be verified programmatically — account-level event subscriptions contain exactly one entry (an unrelated third-party wildcard), and the per-survey workflows API is not readable with the current token. This needs a human in the Qualtrics UI.
Blocked on. Someone opening the Workflows tab on each FY27 survey and checking. Nothing external.
Where. All FY27 surveys; tl-api /update/update_survey_s3; run_all_files.r in ec2_tl_scripts.
analysts_match_27 / project_match_27 do not exist
Unfinished. tl-monday/analysts_match_27.parquet and project_match_27.parquet have never been written, so three consumers stay on the FY26/25_26 objects.
How we know. Checked 2026-08-04. update_coachee_selection_27.qmd carries an inline comment pinning get_analysts_sites(year = "25_26") for this reason.
Blocked on. Whatever produces these objects being pointed at FY27.
Where. tl-api /get_project_analyst; TeachingLab::get_analysts_sites(); TeachingLab::get_staffing_match(); update_coachee_selection_27.qmd.
update_coaching_log_27 and tracker_27 fail by design
Unfinished. Both fail on purpose pending upstream Monday boards. update_coaching_log_27’s source board 18416482214 is named “IN DEV: FY27 Coaching Log” and has zero items. tracker_27 stops on a scaffold guard until the FY27 partnership_info board exists.
How we know. Checked 2026-08-04.
Blocked on. L&R populating board 18416482214, and creating the FY27 partnerships board (same blocker as update_deck_list above).
Where. posit_automations/data_updates/update_coaching_log_27.qmd; posit_automations/tracker_27.qmd.
Confirm the FY27 student work sheet
Unfinished. update_student_work.R reads sheet 1uB4-JiToCLdl1Ws6OrAKQrjAF6_50Q0UZOJ82qDSXcg / “Student Work Scores”. Nobody has confirmed FY27 uses the same sheet rather than a new one.
How we know. Checked 2026-08-04 — flagged as unconfirmed, not investigated since.
Blocked on. L&R confirming.
Where. update_student_work.R (ec2_tl_scripts).