Fixes #7313

We are excited to review your PR.

So we can do the best job, please check:

• There's a descriptive title that will make sense to other developers some time from now.
• There's associated issues. All PR's should have issue(s) associated - unless a trivial self-evident change such as fixing a typo. You can use the format Fixes #nnnn in your description to cause GitHub to automatically close the issue(s) when your PR is merged.
• Your change description explains what the change does, why you chose your approach, and anything else that reviewers should know.
• You have included any necessary tests in the same PR.

Expose Encoder and Decoder properties in TiktokenTokenizer

af97d02

… isntead of using reflection

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.10%. Comparing base (5090327) to head (140aa42).
Report is 44 commits behind head on main.

@@            Coverage Diff             @@
##             main    #7314      +/-   ##
==========================================
+ Coverage   68.87%   69.10%   +0.23%     
==========================================
  Files        1470     1475       +5     
  Lines      274005   277261    +3256     
  Branches    28403    29043     +640     
==========================================
+ Hits       188717   191606    +2889     
- Misses      77970    78374     +404     
+ Partials     7318     7281      -37     

Member

@razshare I replied on the issue. Let's discuss it there first before we continue here. Thanks a lot for your submission. I converted this PR to be draft for now till we finish the discussion.

Author

@dotnet-policy-service agree

Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.