ZJIT: guard_type_failure happens too frequently on lobsters

[k0kubun]

guard_type_failure is the top exit reason on lobsters other than compile_error_register_spill_*. We should investigate why that happens and reduce such exits.

We should:

1. Investigate how guard_type_failure happens on benchmarks. Try --zjit-trace-exits?
2. Address common guard_type_failure patterns.
   • Is call-threshold or num-profiles too low to profile those types?
   • Should we recompile the ISEQ if this happens?

[shopify-github-archiver]

Summary

• Takashi created a GitHub issue with a list of tasks to improve Rails benchmark performance and encouraged the team to pick up unassigned items related to reducing side exits and dynamic sends.
• Aiden is working on optimizations for the --zjit-trace-exits tool, which Takashi noted would be useful for investigating the issues.
• Max identified that type and shape guard failures are likely due to insufficient profiling leading to incorrect monomorphic specialization.
• Max proposed two potential paths to address this: either increasing profiling to reduce monomorphization or implementing a solution for polymorphism.
• For handling polymorphism, Max suggested exploring options like Side-Branch-Based Versioning (SBBV) or run-time polymorphic caching.

Links

Internal

• Rails benchmarks burn-down list

This summary was generated using OpenAI's google:gemini-2.5-pro with a temperature of 0.5.

🧵 Slack Thread

User		Message
	Takashi Kokubun 2025‑09‑30 18:26	I made a burn-down list for speeding up Rails benchmarks: github.com/Shopify/ruby/issues/713 Every issue mentioned under "Reduce side exits" or "Reduce dynamic sends" seems very important to me, but six of them are not assigned at the moment. If you're unsure what to work on next, please consider setting yourself as an assignee to these issues. I wrote up action items in these six tickets. I assume they're already actioanable enough, but feel free to ask me questions. --zjit-trace-exits that Aiden landed today seems useful for many of them.
	Aiden Fox Ivey 2025‑09‑30 18:33	Regarding --zjit-trace-exits , I should have a few optimizations landed for that feature shortly to speed up using it as well as reduce the less useful information it produces.
	Max Bernstein 2025‑09‑30 19:29	Thank you for putting this together and calling this out!
	Max Bernstein 2025‑09‑30 19:35	Type and shape guard failures are a result of monomorphic specialization gone wrong, likely due to some kind of phase change. Perhaps 5 iterations of profiling is not enough. Production compilers tend to wait way more than 30 & profile *many* more than 5 calls.
	Max Bernstein 2025‑09‑30 19:37	That means that either: • we measure more and therefore monomorphise less, leading to sends not being optimized (but remaining in JIT code), or • we come up with some solution for polymorphism I have thrown around SBBV recently, which would allow us to branch to several monomorphic paths in the JIT. Maxime mentioned this some months ago as well. It's not a trivial thing to implement. We could also try to come up with some run-time polymorphic caching thing, which would look different.

---

Takashi Kokubun archived this conversation from ruby-jit at 2025‑09‑30 20:27.
All times are in UTC.

[k0kubun]

Perhaps 5 iterations of profiling is not enough. Production compilers tend to wait way more than 30 & profile many more than 5 calls.

We could also try to come up with some run-time polymorphic caching thing, which would look different.

If profiling more instructions allows us to address these issues on the benchmark with very little overhead, we can just do that.

If the "phase change" happens way later than we want the call threshold to be, we might want to compile the first version once, profile what went wrong in its side exit, and patch the side exit to a second version for that particular exit reason. That way, you need a minimal number of profiles to address all the exit reasons. However, then you also need to either make sure the next block has the same type expectation and the register state, or just generate new code.

That means that either:

• we measure more and therefore monomorphise less, leading to sends not being optimized (but remaining in JIT code), or

I would suggest doing this until we have the support of polymorphism. Leading to a send not being optimized in the single profiled case seems no worse than giving up any optimization in all non-profiled cases.

• we come up with some solution for polymorphism

I think it's possible to have multiple branches for different profiled types in a single callsite even today. We could create a basic block for each branch and let them jump to the same successor basic block. Thanks to basic block arguments, the (non-CFG-aware) register allocator should still work, and the type can still be specialized if these branches return the same type.

If the return types differ among different branches and we want to have multiple versions for these different types to avoid losing types, we could do basic block versioning, but that seems like a later prioritization.

[tekknolagi]

Leading to a send not being optimized in the single profiled case seems no worse than giving up any optimization in all non-profiled cases.

Maybe! We have to measure. It could be that the counterfactual of a bunch of side exits is we lose an optimization on a really hot function. Hard to know without measuring.

I think it's possible to have multiple branches for different profiled types in a single callsite even today.

True--I agree. This is sort of a limited form of SBBV with path merging, or something. It would be an interesting experiment.

[st0012]

The highest jit_entry_calls I got after lobsters' first iteration is 111464.

So theoretically, ruby --zjit-stats --zjit-num-profiles=111463 --zjit-call-threshold=111464 -I./harness benchmarks/lobsters/benchmark.rb should have 0 guard_type_failure, but this is the stats I got:

***ZJIT: Printing ZJIT statistics on exit***
Top-20 unoptimized sends to C functions (97.6% of total 3,722,562):
                       Class#current: 1,687,737 (45.3%)
           CGI::EscapeExt#escapeHTML:   206,688 ( 5.6%)
                       TrueClass#===:   181,369 ( 4.9%)
                      FalseClass#===:   178,789 ( 4.8%)
                       String#empty?:   177,691 ( 4.8%)
                  String#ascii_only?:   177,539 ( 4.8%)
  ActiveSupport::OrderedOptions#_get:   135,950 ( 3.7%)
                             Hash#[]:   119,582 ( 3.2%)
                     String#include?:   115,579 ( 3.1%)
                         Integer#<=>:    94,996 ( 2.6%)
                       Range#member?:    92,956 ( 2.5%)
                      Array#include?:    86,675 ( 2.3%)
                        NilClass#===:    57,185 ( 1.5%)
                      BasicObject#!=:    55,062 ( 1.5%)
                     Kernel#kind_of?:    53,560 ( 1.4%)
            SQLite3::Statement#done?:    48,618 ( 1.3%)
             SQLite3::Statement#step:    48,618 ( 1.3%)
                        Set#include?:    47,310 ( 1.3%)
                           String#==:    38,987 ( 1.0%)
          Time#plus_without_duration:    26,721 ( 0.7%)
Top-4 not optimized method types (100.0% of total 669,193):
       iseq: 339,860 (50.8%)
  optimized: 167,898 (25.1%)
      alias: 115,579 (17.3%)
    bmethod:  45,856 ( 6.9%)
Top-8 not optimized instructions (100.0% of total 10,367,080):
                opt_aref: 5,914,075 (57.0%)
                    send: 3,343,641 (32.3%)
             invokesuper:   538,720 ( 5.2%)
                  opt_eq:   323,663 ( 3.1%)
             invokeblock:   214,372 ( 2.1%)
  opt_send_without_block:    14,573 ( 0.1%)
                opt_plus:    14,573 ( 0.1%)
             sendforward:     3,463 ( 0.0%)
Top-5 send fallback reasons (100.0% of total 26,540,577):
                     not_optimized_instruction: 10,367,080 (39.1%)
                send_without_block_polymorphic:  9,788,320 (36.9%)
         send_without_block_cfunc_not_variadic:  3,722,562 (14.0%)
                send_without_block_no_profiles:  1,993,422 ( 7.5%)
  send_without_block_not_optimized_method_type:    669,193 ( 2.5%)
Top-2 unhandled YARV insns (100.0% of total 361,053):
   expandarray: 337,465 (93.5%)
  checkkeyword:  23,588 ( 6.5%)
Top-1 compile error reasons (100.0% of total 1,321,885):
  register_spill_on_alloc: 1,321,885 (100.0%)
Top-9 side exit reasons (100.0% of total 2,988,572):
                        compile_error: 1,321,885 (44.2%)
  block_param_proxy_not_iseq_or_ifunc:   467,493 (15.6%)
                unknown_newarray_send:   405,518 (13.6%)
                  unhandled_yarv_insn:   361,053 (12.1%)
                  guard_shape_failure:   232,056 ( 7.8%)
                   guard_type_failure:   168,114 ( 5.6%)
                      unhandled_kwarg:    20,488 ( 0.7%)
                   unhandled_hir_insn:    11,912 ( 0.4%)
                            interrupt:        53 ( 0.0%)
dynamic_send_count:                   26,540,577
dynamic_getivar_count:                8,710,593
dynamic_setivar_count:                855,633
compiled_iseq_count:                  177
failed_iseq_count:                    9
compile_time:                         266ms
profile_time:                         73,240ms
gc_time:                              61ms
invalidation_time:                    0ms
vm_write_pc_count:                    32,183,689
vm_write_sp_count:                    32,043,692
vm_write_locals_count:                32,043,692
vm_write_stack_count:                 32,043,692
vm_write_to_parent_iseq_local_count:  21,915
vm_read_from_parent_iseq_local_count: 3,856,818
code_region_bytes:                    671,744
side_exit_count:                      2,988,572
total_insn_count:                     912,759,398
vm_insn_count:                        741,298,904
zjit_insn_count:                      171,460,494
ratio_in_zjit:                        18.8%

[k0kubun]

We paired on investigating this today. With --zjit-trace-exits=exit_guard_type_failure,

$ stackprof benchmarks/lobsters/zjit_exits_330147.dump
==================================
  Mode: ()
  Samples: 529994 (0.00% miss rate)
  GC: 0 (0.00%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
    341433  (64.4%)      341433  (64.4%)     opt_send_without_block
    122547  (23.1%)      122547  (23.1%)     opt_nil_p
     26203   (4.9%)       26203   (4.9%)     opt_eq
     24976   (4.7%)       24976   (4.7%)     opt_or
     11442   (2.2%)       11442   (2.2%)     opt_not
      1558   (0.3%)        1558   (0.3%)     opt_neq
       648   (0.1%)         648   (0.1%)     opt_ltlt
       603   (0.1%)         603   (0.1%)     opt_empty_p
       229   (0.0%)         229   (0.0%)     opt_and
       187   (0.0%)         187   (0.0%)     opt_aset
       158   (0.0%)         158   (0.0%)     newrange
        10   (0.0%)          10   (0.0%)     opt_aref

$ stackprof benchmarks/lobsters/zjit_exits_330147.dump --method opt_send_without_block
opt_send_without_block (nonexistent.def:1)
  samples:  341433 self (64.4%)  /   341433 total (64.4%)
  callers:
    33433  (    9.8%)  ActiveRecord::Base._reflections
    26722  (    7.8%)  ActiveModel::Type::Value#deserialize
    24482  (    7.2%)  ActiveRecord::Base#__callbacks
    16850  (    4.9%)  Gem::Dependency#matches_spec?
    12540  (    3.7%)  TZInfo::TransitionRule#at
    12540  (    3.7%)  TZInfo::AnnualRules#apply_rule
    12212  (    3.6%)  ActiveRecord::Core#init_internals
    12212  (    3.6%)  ActiveRecord::Base.strict_loading_mode
    12212  (    3.6%)  ActiveRecord::Base.strict_loading_by_default
    12177  (    3.6%)  ActiveRecord::Core#init_with_attributes
    12177  (    3.6%)  ActiveRecord::Persistence::ClassMethods#instantiate_instance_of
   ...

As an example, we looked at exits in ActiveModel::Type::Value#deserialize (because ActiveRecord::Base._reflections was meta programming and didn't show nice source annotations).

$ stackprof benchmarks/lobsters/zjit_exits_330147.dump --method 'ActiveModel::Type::Value#deserialize'
ActiveModel::Type::Value#deserialize (/opt/rubies/ruby/lib/ruby/gems/3.5.0+4/gems/activemodel-8.0.3/lib/active_model/type/value.rb:43)
  samples:     0 self (0.0%)  /   108386 total (20.5%)
  callers:
    52957  (   48.9%)  ActiveModel::LazyAttributeSet#fetch_value
    27943  (   25.8%)  ActiveRecord::Result#cast_values
    27077  (   25.0%)  block in delegating_block
     409  (    0.4%)  ActiveModel::Attribute::FromDatabase#type_cast
  callees (108386 total):
    67567  (   62.3%)  ActiveModel::Type::Value#cast
    26722  (   24.7%)  opt_send_without_block
    14097  (   13.0%)  ActiveModel::Type::Helpers::AcceptsMultiparameterTime::InstanceMethods#cast
  code:
                                  |    43  |       def deserialize(value)
 108386   (20.5%)                 |    44  |         cast(value)
                                  |    45  |       end

I modified #deserialize to print the receiver type for the #cast call that's causing exits:

       def deserialize(value)
+        p self.class
         cast(value)
       end

The result was:

ActiveModel::Type::String
ActiveModel::Type::String
ActiveRecord::Type::DateTime
ActiveRecord::Type::DateTime
ActiveRecord::Type::Text
ActiveModel::Type::String
ActiveModel::Type::String
ActiveModel::Type::String
ActiveModel::Type::String
ActiveModel::Type::String
...(26000-ish entries of ActiveModel::Type::String)...
ActiveModel::Type::Boolean
ActiveRecord::Type::DateTime
ActiveModel::Type::Boolean
ActiveRecord::Type::DateTime
ActiveModel::Type::Boolean
ActiveRecord::Type::DateTime
ActiveModel::Type::Boolean
ActiveRecord::Type::DateTime
ActiveModel::Type::Boolean
ActiveRecord::Type::DateTime
ActiveModel::Type::Boolean
ActiveRecord::Type::DateTime
...(alternates Boolean and DateTime forever)...

So there's a phase change. We discussed this a little and decided to experiment with recompiling the whole ISEQ, after modifying profiling data on the exit.

If we clear jit_entry and set 0 (or maybe 25?) to jit_entry_calls, it would run multiple profile iterations that observe both Boolean and DateTime (besides the profile captured on exit), so it should successfully observe all three types at the second compilation for this callsite.

[k0kubun]

Filed #847

[alanwu-shopify-inc]

I believe this will be mostly solved in lobsters after ruby#17133, with the ratio going up to 97%. Feel free to unlink the issue if you disagree.

[jacob-shops]

Thanks for linking Alan!