feat: Add survey completion time tracking and display for analytics.

[lakshita10341]

What this PR does

Adds survey completion duration tracking so admins can see how long users take to complete surveys.
Closes #6355
Problem: The survey system captures what users answered but has no way to measure how long each submission takes. This makes it difficult to identify surveys that are too long or confusing.

Solution: A new survey_completion_times table records start/end timestamps for each survey submission. The frontend sends a POST when the user starts the survey, and a PUT when they submit. Duration is computed server-side.

Changes

Backend:

• New migration, model (SurveyCompletionTime), and controller with start/complete actions
• New routes: POST /survey/start and PUT /survey/complete
• CSV export (Survey#to_csv) now includes duration_seconds as the last column

Frontend:

• Survey.js sends tracking requests on survey start and completion (fails silently to never block the survey)

Admin views:

• Survey list and results index show "Avg Duration" column
• Per-survey results page shows full stats: average, median, fastest, slowest, completion rate, and a time distribution chart

Edge cases handled:

• Old surveys show -- for all stats (no backfill needed)
• Abandoned surveys (started but not finished) are excluded from stats

AI usage

I used Gemini to help implement this feature. The AI was used for:

• Planning the database schema
• Writing the migration, RSpec model and controller tests
• Debugging Docker environment issues and data setup for local testing

Screenshots

Before:

After:

##Open concerns

• Duration tracking only applies to surveys taken after this change is deployed. There is no backfill for historical survey submissions.

[Abishekcs]

The frontend sends a POST when the user starts the survey, and a PUT when they submit. Duration is computed server-side.

The frontend sends a POST when the user starts the survey - This will create a record in the DB right ? So what if the user lefts the survey before they submit.

I haven't look at the code yet so I had this question.

[Abishekcs]

duration_in_seconds is this required ? Can't the time to calculate SurveyCompletionTime be obtained from started_at and completed_at

[lakshita10341]

yes, sorry i missed it. The duration column is not required, so i will remove it in next commit.

[lakshita10341]

The frontend sends a POST when the user starts the survey, and a PUT when they submit. Duration is computed server-side.

The frontend sends a POST when the user starts the survey - This will create a record in the DB right ? So what if the user lefts the survey before they submit.

I haven't look at the code yet so I had this question.

Yes, this will create a record in db, with some starting time and 'nil' ending time if the user does not submit the survey. I added a scope :completed, -> { where.not(completed_at: nil) } to the model, so all of our duration statistics (average, median, fastest, etc.) explicitly filter out those abandoned surveys and only look at fully completed ones and additionally this also allow us to display the completion rate which we our showing at admin frontend.

[ragesoss]

I don't think it makes sense to add a separate table for survey completion time. We already have a table that represents a set of survey responses being submitted, which is Rapidfire::AnswerGroup. I think adding a started_at field to that table be sufficient; created_at represents the timestamp when the survey was completed.

[lakshita10341]

I don't think it makes sense to add a separate table for survey completion time. We already have a table that represents a set of survey responses being submitted, which is Rapidfire::AnswerGroup. I think adding a started_at field to that table be sufficient; created_at represents the timestamp when the survey was completed.

That is right, this approach will keep the database clean. But the main reason I initially considered a separate table was to track abandonment rates and avoid data duplication issues.

Since AnswerGroup records are only created after a successful submission, we wouldn't be able to see how many people started a survey but quit halfway through. The separate table allows us to calculate the Completion Rate analytics (starts vs. completions).

Also, since a Survey can contain multiple QuestionGroups, submitting a single survey often creates multiple AnswerGroup rows (one for each group). So, we need to duplicate the started_at timestamp across all of them.

Please let me know, if adding field to Rapidfire::AnswerGroup table is preferred approach, I'm happy to proceed with that!

[ragesoss]

That's a good point about tracking abandonment rates, and I think I'm open to a new table if that's part of the design. The name of such a table should be clearer, as survey_completion_times implies it's only for completed surveys.

[lakshita10341]

Will SurveySession work better?

[ragesoss]

That sounds reasonable, although 'session' implies each record is limited to a single session, with different records if a user starts and abandons a survey, then later starts and completes it. Would that be how it works?

[lakshita10341]

At every 'Start' attempt, there will be a new session record created, which lets us accurately track abandonment vs. completion rates. For the CSV export and duration analytics, we then only look at the successfully completed session.

[Abishekcs]

Is index on all three required ?

add_index :survey_sessions, :survey_id
add_index :survey_sessions, :user_id
add_index :survey_sessions, %i[survey_id user_id]

[lakshita10341]

Is index on all three required ?

add_index :survey_sessions, :survey_id
add_index :survey_sessions, :user_id
add_index :survey_sessions, %i[survey_id user_id]

Yeah, Since we have composite index on [:survey_id, :user_id], we can remove the survey_id index. I'll update the code.

[Abishekcs]

Can you also point to which part of the code is using this composite index

[lakshita10341]

Can you also point to which part of the code is using this composite index

 add_index :survey_sessions, %i[survey_id user_id]

I am referring to this part of code

[Abishekcs]

No I mean in the actual Survey codebase

[Abishekcs]

That might be confusing what I just said.

I mean which Query you wrote is utilizing this Composite Index of survey_id and user_Id.

[Abishekcs]

One problem with adding indexes early is cardinality. Make sure you’ve checked in MySQL that the query planner is actually using both indexes.

[lakshita10341]

One problem with adding indexes early is cardinality. Make sure you’ve checked in MySQL that the query planner is actually using both indexes.

While the query is not using directly both the indexes, but this composite index still helps here:

SurveySession.where(survey_id: id).index_by(&:user_id)

Although the survey_id has low cardinality, but this composite index is acting as covering index mapping directly the user id from index tree, making the lookup fast.

[Abishekcs]

hmm can you show me the query planner using EXPLAIN or which ever you prefer

[Abishekcs]

I don't think the user_id is being even used

[lakshita10341]

Sorry I was wrong. Since we are using select * the composite index here have no use. So we can remove this, and can keep index on survey_id only. Will it work?

[Abishekcs]

Sorry I was wrong. Since we are using select * the composite index here have no use. So we can remove this, and can keep index on survey_id only. Will it work?

keeping index on only survey_id make sense.

[Abishekcs]

What is the purpose of optional here ?

[lakshita10341]

If the user takes a survey without clicking on the email link, then survey notification id will be null. That's why optional is used.

[Abishekcs]

After removing it from Composite Index why add it separately again? Where is this being used ?

[lakshita10341]

Sorry, I meant to remove it alongside the composite index but it was an oversight. I've removed it now. I realized we can add it back if in future we need it.

[Abishekcs]

Thanks for the changes. I will have a look within the next few days,