Trigger ios-usb-dependencies builder to run weekly

Run ios-usb-dependencies builder weekly.
This uploads the iOS vendored artifacts to an ios-usb-dependencies/unsigned bucket, which is not used by the Flutter tool (still needs to be codesigned, validated, and uploaded to the signed bucket.  See go/flutter-manual-codesign and ./codesign.py --ios-deploy (maybe?) and https://flutter-review.googlesource.com/c/recipes/+/32660/
So I was wrong in https://flutter-review.googlesource.com/c/infra/+/32700/comment/9f5b198c_272b6c0d/, I believe we should run this more often.
cc @keyonghan

https://flutter-review.googlesource.com/c/infra/+/34161 to bump to weekly.

Closing as it looks like this was addressed.


Add support for $addToSet array update operator

Description
Closes #825.
I didn't add any test cases with array values, as they would result in embedded arrays. Also can't come up with any invalid name that would return error from MongoDB.

Readiness checklist

[ ] I added/updated unit tests.
[x] I added/updated integration/compatibility tests.
[x] I added/updated comments and checked rendering.
[x] I made spot refactorings.
[x] I updated user documentation.
[ ] I ran task all, and it passed.
[x] I ensured that PR title is good enough for the changelog.
[x] (for maintainers only) I set Reviewers (@FerretDB/core), Assignee, Labels, Project and project's Sprint fields.
[x] I marked all done items in this checklist.

@w84thesun I see Tigris fails for addToSet tests, it is worth checking why (feel free to ping me if something is unclear about Tigris behavior).


Fail create post

I have tried, nothing is modified to just post one blogpost and i have this error :
` Microsoft.EntityFrameworkCore.Database.Command[20101]
Executed DbCommand (2ms) [Parameters=[@p0='?' (DbType = Int32), @p1='?' (Size = 4000), @p2='?' (DbType = DateTime2), @p3='?' (Size = 4000), @p4='?' (Size = 500), @p5='?' (DbType = Boolean), @p6='?' (DbType = Boolean), @p7='?' (DbType = Boolean), @p8='?' (DbType = DateTime2), @p9='?' (Size = 125), @p10='?' (Size = 100), @p11='?' (Size = 450), @p12='?' (DbType = Int32)], CommandType='Text', CommandTimeout='30']
SET IMPLICIT_TRANSACTIONS OFF;
SET NOCOUNT ON;
INSERT INTO [BlogPosts] ([CategoryId], [Content], [CreatedAt], [Image], [Introduction], [IsDeleted], [IsFeatured], [IsPublished], [PublishedAt], [Slug], [Title], [UserId], [ViewCount])
OUTPUT INSERTED.[Id]
VALUES (@p0, @p1, @p2, @p3, @p4, @p5, @p6, @p7, @p8, @p9, @p10, @p11, @p12);
fail: Microsoft.EntityFrameworkCore.Update[10000]
An exception occurred in the database while saving changes for context type 'BlazorBlog.Data.ApplicationDbContext'.
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.
---> Microsoft.Data.SqlClient.SqlException (0x80131904): Cannot insert the value NULL into column 'Image', table 'BlazorBlog.dbo.BlogPosts'; column does not allow nulls. INSERT fails.
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction)          at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction)
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows)
at Microsoft.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more)
at Microsoft.Data.SqlClient.SqlDataReader.ReadAsyncExecute(Task task, Object state)
at Microsoft.Data.SqlClient.SqlDataReader.InvokeAsyncCall[T](SqlDataReaderBaseAsyncCallContext1 context)       --- End of stack trace from previous location ---
A problem with the image upload it's look like

Hello,
Thank you for reporting this issue. Based on the error description, it seems that the attempt to insert a record into the BlogPosts table is failing because the Image column value is NULL, which causes an error as this column does not allow NULL values.
One way to address this problem is by modifying the BlogPost model to include Data Annotations that enforce the Image as a mandatory field. I suggest adding the following attributes to the Image property in your model:
[Required(ErrorMessage = "The image is required.")]
[MaxLength(100, ErrorMessage = "The length of the image URL cannot exceed 100 characters.")]
public string Image { get; set; }

With these changes applied, the application will require users to provide an image. This should prevent the error you are encountering.
Please try this solution and let me know if it resolves the issue.

"_fileToUpload" is always null, even when i upload a file, thats the problem.
When i try for ex an url (bypassing if (_fileToUpload is not null)) it is working with the url.
I gonna search more deep tomorow
 private async Task HandleFileUploadAsync(InputFileChangeEventArgs e)  {      Console.WriteLine("Nombre d'image : "+ e.FileCount.ToString());      await PreviewImageAsync(e.File);      _fileToUpload = e.File;<=== return nothing  }
I think it's the problem, i have 1 count but it is not working.
Sory for my bad English.

Ok so easy enough, wrapping the Task "PreviewImageAsync" with IOException i find out that the maximum file upload is set by defaut on 500kb... I feel dumb right now.
i just adjust imageStream limitation size to in my case is 10mb for ex :
using var imageStream = file.OpenReadStream(maxAllowedSize:10000000)
End it's working like a charm.

Hello,
Thank you for sharing the details regarding the issues you've encountered. I wanted to inform you that I have made several changes to the code that address these concerns. These changes include improvements to the SaveBlogPost component for more efficient content and image handling, as well as optimization of the file upload process. These updates have already been uploaded to the repository.
Additionally, I have taken into account your remark about the size limitation of image files. I have implemented a fix that increases the maximum size of uploaded files to 10MB. This should enhance functionality and provide more flexibility when working with images. Appropriate user messages have also been included to inform about this limit to avoid any confusion.
Please review the changes and provide your feedback. Your insights and suggestions are valuable for the continuous improvement of the project.
Thank you again for your contribution and support.
Best regards,
unrealbg

Hello you need also a change in Task SaveFileAsync because it cause the same issue (again "OpenReadStream"):
I have created a public ref :
public long MaxFileLenght = 10 * 1024 * 1024;
For saving the file :
 await file.OpenReadStream(maxAllowedSize: MaxFileLenght).CopyToAsync(fs);
For the preview i use MaxFileLenght to.
For the preview you have already do the change nice, i see more check for errors and some improvement in new version thats great !
No problem 👍

Hello,
Thank you for pointing out the need for a change in the SaveFileAsync method in relation to the file size limitation issue. I appreciate your vigilance in identifying that this method also requires the same adjustment as the PreviewImageAsync method.
I have implemented the suggested modification:
private async Task<string?> SaveFileAsync(IBrowserFile file)
{
    var webRootPath = WebHostEnvironment.WebRootPath;
    var folderPath = Path.Combine(webRootPath, "images", "posts");
    Directory.CreateDirectory(folderPath);

    var randomFileName = Path.GetRandomFileName();
    var extension = Path.GetExtension(file.Name);
    var filePath = Path.Combine(folderPath, randomFileName + extension);

    await using FileStream fs = new(filePath, FileMode.Create);

    try
    {
        await file.OpenReadStream(maxAllowedSize: MaxFileLenght).CopyToAsync(fs);
        return Path.Combine("images", "posts", randomFileName + extension).Replace("\\", "/");
    }
    catch (Exception)
    {
        ToastService.ShowToast(ToastLevel.Error, "Something went wrong while saving the file.", heading: "Error");
        fs.Close();
        return null;
    }
}

I've now incorporated the MaxFileLenght constant to both the SaveFileAsync and PreviewImageAsync methods, ensuring consistency across the file handling process.
I'm glad to hear that the other improvements and error checks in the new version are to your satisfaction. Your feedback is invaluable in making our project more robust and user-friendly.
Please do not hesitate to reach out if there are any more suggestions or issues.
Best regards,
unrealbg


Headers timeout from undici

Sometimes we get "Headers Timeout Error" from undici which is being used here, is there any solution you could implement for this, or an alternative to undici that could be used?

Currently i dont know why this happens, i read that its a node problem
Callum OKane @.***> schrieb am So., 28. Mai 2023, 00:41:

Sometimes we get "Headers Timeout Error" from undici which is being used
here, is there any solution you could implement for this, or an alternative
to undici that could be used?
—
Reply to this email directly, view it on GitHub
https://github.com/Tomato6966/erela.js/issues/24, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQH5DI2IUWZEQSRJKZCQSV3XIJ7JPANCNFSM6AAAAAAYRN6D4Q
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Fix some robot stuff

Coverage increased (+0.0003%) to 92.134% when pulling 261d7fefb14b78d2629ac53626656bb659aa6be2 on fix-some-robot-stuff into 15c1d3b4df6d6f16ae8282fbc306d9a8eafcdaaa on master.


Lay the groundwork for #25

Update gradle file to deploy to Sonatype (conduit to maven central)

Codecov Report

Merging #43 into development will not change coverage.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             development      #43   +/-   ##
==============================================
  Coverage          56.38%   56.38%
  Complexity           159      159
==============================================
  Files                 57       57
  Lines               1002     1002
  Branches              94       94
==============================================
  Hits                 565      565
  Misses               388      388
  Partials              49       49

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c366fc1...2594341. Read the comment docs.

I'm going to close this PR and re-open once it's correct.  The crypto-condition project is further along, so once that's working, I'll re-open this.


feat: empty-response event to handle no server responses

Description
Make a emtpy-response to handle no server responses with a param named counts which is the number of no server responses appears consecutively.
Motivation and Context
fix #228 Handling ZNOTHING inside may not be a good idea
How Has This Been Tested?
Changed and ran example.

Nice! I'll have a look and will give you feedback this weekend.


Updates to min specilazation

rustc 1.71 introduced some changes to min specialization
https://github.com/rust-lang/rust/pull/111252
We will remove min_specialization altogether, so we will no longer depend on nightly features, as they may introduce breaking changes to a stable product. However, this would remove some of OpenBrush's unique features, which must be implemented differently.
TODO:

[ ] Remove min specialization
[ ] Test it
[ ] Reusable traits
[ ] add implementation macro, which will implement all needed traits
[ ] add override macro, which will handle overriding of methods
[ ] Test it

Will this makes OB compile in stable ?

yes)

How do you plan on supporting default fns in default impls?

Hi @deuszx, we are planning to use a bit another approach for it,  with use of macros


flopper

mod intergration with flopper, if container is almost full it removes the bucket when right clicking to fill

https://github.com/CyclopsMC/Flopper/issues/14


segfault in node_set.rb

Describe the bug
/home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node_set.rb:238: [BUG] Segmentation fault at 0x000000000a296680
ruby 2.7.0dev (2019-11-28T14:49:28Z trunk b5fbefbf2c) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0011 p:---- s:0048 e:000047 CFUNC  :[]
c:0010 p:0006 s:0043 e:000042 BLOCK  /home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node_set.rb:238 [FINISH]
c:0009 p:---- s:0039 e:000038 CFUNC  :upto
c:0008 p:0019 s:0034 e:000033 METHOD /home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node_set.rb:237
c:0007 p:0037 s:0030 e:000029 METHOD /home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node.rb:261
c:0006 p:0036 s:0025 e:000023 BLOCK  seg.rb:13 [FINISH]
c:0005 p:---- s:0021 e:000020 CFUNC  :times
c:0004 p:0055 s:0017 e:000016 BLOCK  seg.rb:12 [FINISH]
c:0003 p:---- s:0010 e:000009 CFUNC  :times
c:0002 p:0014 s:0006 e:000005 EVAL   seg.rb:7 [FINISH]
c:0001 p:0000 s:0003 E:002200 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
seg.rb:7:in `<main>'
seg.rb:7:in `times'
seg.rb:12:in `block in <main>'
seg.rb:12:in `times'
seg.rb:13:in `block (2 levels) in <main>'
/home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node.rb:261:in `children='
/home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node_set.rb:237:in `each'
/home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node_set.rb:237:in `upto'
/home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node_set.rb:238:in `block in each'
/home/ahorek/.rvm/gems/ruby-head/gems/nokogiri-1.10.5/lib/nokogiri/xml/node_set.rb:238:in `[]'
...

To Reproduce
require 'nokogiri'

def roundup(num, nearest = 10)
  num % nearest == 0 ? num : num + nearest - (num % nearest)
end

1000.times do |i|
  response = File.read('xml.xml')
  response2 = response.dup
  x = Nokogiri::XML.parse(response)
  x.root.content = ''
  140.times do
    x.root.children += Nokogiri::XML.parse(response2).root.children
  end
  puts i
end
puts 'ok!'

xml input
https://github.com/ahorek/nokogiri/blob/bug/bug/xml.xml
Expected behavior
no segfaults
Environment
latest nokogiri-1.10.5
ruby 2.7.0dev (2019-11-28T14:49:28Z trunk b5fbefbf2c) [x86_64-linux]
tested on multiple ruby versions 2.5 - 2.7, linux / windows
the script doesn't fail on jruby

Hi, thank you for reporting this. I'll attempt to reproduce and diagnose as soon as I can (hopefully today).

OK, I've reproduced using your script and XML. Currently trying to reproduce that while running under Valgrind to get more precise information about the operation causing the segfault.

OK - I captured the segfault in valgrind and have a nice stack walkback. Will dig in.

Here's the top of the valgrind output, cleaned up a bit for clarity:
==27690== Invalid read of size 4
==27690==    at 0xB37CED0: Nokogiri_wrap_xml_node_set_node (xml_node_set.c:424)
==27690==    by 0xB37D584: index_at (xml_node_set.c:236)
==27690==    by 0xB37D584: slice (xml_node_set.c:312)
==27690==    by 0x50924E9: vm_call_cfunc_with_frame (vm_insnhelper.c:1908)
==27690==    by 0x50924E9: vm_call_cfunc (vm_insnhelper.c:1924)
==27690==    by 0x509BEF1: vm_exec_core (insns.def:765)
==27690==    by 0x50A290E: rb_vm_exec (vm.c:1885)
...
==27690==  Address 0xee0e5b8 is 8 bytes inside a block of size 120 free'd
==27690==    at 0x4C30D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27690==    by 0x4F19EDC: objspace_xfree (gc.c:8309)
==27690==    by 0x4F19EDC: ruby_sized_xfree (gc.c:8405)
==27690==    by 0x4F19EDC: ruby_xfree (gc.c:8412)
==27690==    by 0xB3CE9EB: xmlFreeNodeList (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3CAA11: xmlFreeDoc (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0x4F10BF0: run_final (gc.c:2917)
==27690==    by 0x4F10BF0: finalize_list (gc.c:2936)
==27690==    by 0x4F10CE7: finalize_deferred (gc.c:2956)
==27690==    by 0x4F10CE7: gc_finalize_deferred (gc.c:2965)
...
==27690==  Block was alloc'd at
==27690==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27690==    by 0x4F1C8F3: objspace_xmalloc0 (gc.c:8134)
==27690==    by 0xB4C2144: xmlSAX2TextNode (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB4C3BBF: xmlSAX2Characters (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3AB1E4: xmlParseCharData (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3BB282: xmlParseContent (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3BB3D1: xmlParseElement (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3BDD86: xmlParseDocument (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3C78F9: xmlDoRead (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3C7A90: xmlReadMemory (in /home/flavorjones/.rvm/gems/ruby-2.6.2/gems/nokogiri-1.10.5/lib/nokogiri/nokogiri.so)
==27690==    by 0xB3785E4: read_memory (xml_document.c:294)
==27690==    by 0x50924E9: vm_call_cfunc_with_frame (vm_insnhelper.c:1908)
==27690==    by 0x50924E9: vm_call_cfunc (vm_insnhelper.c:1924)
==27690==    by 0x509BEF1: vm_exec_core (insns.def:765)
==27690==    by 0x50A290E: rb_vm_exec (vm.c:1885)
...

I need to investigate, but essentially what this is telling me is that we still have an active pointer to memory that was `free()`ed by libxml2. And that's bad.

Sorry for lack of updates - this is high priority, but I haven't been able to spend much time on it in the past week. (The difficulty in quickly/reliably reproducing it is an additional challenge.)

I've encountered this same exact issue with nokogiri 1.10.9 and ruby 2.6.3

@samsonnguyen I'm sorry you're experience this issue!
Are you able to easily reproduce this? It would greatly help if I had a repro that I could use to get faster feedback loops.

@ahorek Thanks for the PR with the repro, I'll dig in.

I have come across a segmentation fault which I think is related to this issue. It happens when I merge two nodesets like this:
require "nokogiri"

XML_PAYLOAD = <<-EOF
    <?xml version="1.0" encoding="UTF-8"?>
    <container>
    </container>
EOF

class ParseAndMerge
    def merge_items
        nodeset1 = Nokogiri::XML(XML_PAYLOAD).xpath("//container")
        nodeset2 = Nokogiri::XML(XML_PAYLOAD).xpath("//container")
        nodeset1 + nodeset2
    end
end

GC.stress = true
100.times do
    p = ParseAndMerge.new
    merged_nodeset = p.merge_items
    puts merged_nodeset.to_s
end

Happens with 1.10.9, 1.10.10, 1.11.0.rc3. nokogiri -v:
  nokogiri_segfault_test nokogiri -v
# Nokogiri (1.11.0.rc3)
    ---
    warnings: []
    nokogiri: 1.11.0.rc3
    ruby:
      version: 2.6.6
      platform: x86_64-darwin19
      gem_platform: x86_64-darwin-19
      description: ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-darwin19]
      engine: ruby
    libxml:
      source: packaged
      patches:
      - 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
      - 0002-Remove-script-macro-support.patch
      - 0003-Update-entities-to-remove-handling-of-ssi.patch
      - 0004-libxml2.la-is-in-top_builddir.patch
      - 0005-Fix-infinite-loop-in-xmlStringLenDecodeEntities.patch
      compiled: 2.9.10
      loaded: 2.9.10
    libxslt:
      source: packaged
      patches: []
      compiled: 1.1.34
      loaded: 1.1.34

It does not happen when I change above code to prevent ruby from garbage collecting the nodeset2 (nodeset1.+(nodeset2)), like this:
require "nokogiri"

XML_PAYLOAD = <<-EOF
    <?xml version="1.0" encoding="UTF-8"?>
    <container>
    </container>
EOF

$nodeset_list = []
class ParseAndMerge
    def merge_items
        nodeset1 = Nokogiri::XML(XML_PAYLOAD).xpath("//container")
        nodeset2 = Nokogiri::XML(XML_PAYLOAD).xpath("//container")
        $nodeset_list << [nodeset2] # Fix: Prevent ruby from garbage collecting nodeset2
        nodeset1 + nodeset2
    end
end

GC.stress = true
100.times do
    p = ParseAndMerge.new
    merged_nodeset = p.merge_items
    puts merged_nodeset.to_s
end

@stayhero Thanks for reporting your issue. I don't have enough information yet to determine if you're seeing the same problem or not. Hoping to be able to dedicate some time to this memory issue soon.

@stayhero Sorry for the delay in giving you a meaningful response. After digging into the example you've given, I'm confident that this is the same bug. Thanks so much for providing this second example which has helped illuminate what's going on.

@stayhero Sorry for the delay in giving you a meaningful response. After digging into the example you've given, I'm confident that this is the same bug. Thanks so much for providing this second example which has helped illuminate what's going on.

I'm going to try to explain how GC works with libxml2's data structures, what's going wrong in these examples to cause a segfault, and some ideas for resolving the problems.
How GC works for libxml2 data structures
Garbage collection ("GC") of libxml2 data structures is controlled by the Document object's lifecycle -- the libxml2 function xmlFreeDoc frees all the node data structures owned by the document, and so we take some pains to make sure that Document and its related Node objects are all GCed at the same time.
The Document object keeps a reference to every Node object in a "node cache", even if the node has been unparented from the DOM tree. Every Node object keeps a reference back to its parent Document in the @document attribute. So if either the Document or any of its Nodes is still referenced, then their mutual references to each other keep them from being GCed.
Additionally, a NodeSet keeps a reference to its Document, too -- this is a libxml2 implementation detail that doesn't have any major implications to the Nokogiri user (primarily, it's the module decorators, which allow library users to extend the Node and NodeSet interface). But it's worth noting that if a NodeSet is still referenced, then its reference to the Document keeps that document alive.
Notably, what NodeSet does not do is mark each of the nodes it contains during the GC "mark" phase. This may be surprising to you if you know how all the standard Ruby collection classes work. This behavior was adopted for NodeSet because we assume that it only contains Nodes belonging to the same Document as itself, and so marking is unnecessary and would only slow GC.
What's Going Wrong
In these examples, we have two sets of Nodes, each belonging to a different parent Document and contained by two separate NodeSets. The NodeSets are merged using #+ which invokes the set_union function in xml_node_set.c. This function creates a new xmlNodeSet associated with Document of the lhs's NodeSet, and adds the Nodes from the rhs into it. So now we have a NodeSet referencing only Document A which contains Nodes from both Document A and Document B.
I mentioned above that NodeSet does not mark the Nodes it contains because it assumes they all belong to the same Document as itself, and so marking is unnecessary. In these examples, that's an invalid assumption.
In these examples, when GC kicks in, nothing belonging to Document B is marked -- there are no references to Document B or its NodeSet, and the only references to the Nodes are in the NodeSet belonging to Document A, which does not mark them during GC. As a result, Document B, NodeSet B, and all the nodes belonging to it are mistakenly considered "no longer referenced" and GCed and freed.
Later, we try to print out the contents of the NodeSet and dereference invalid pointers, causing a segfault.
Thoughts on Resolution
Two solutions come to mind:

ensure that a NodeSet only contains Nodes parented by the its Document, duplicating the Node when necessary to ensure this property
ensure that NodeSets mark their contents, similar to Array and other standard Ruby container classes.

The disadvantages of option (2) are:

in edge cases it has the potential to keep references alive to many Documents and preventing them -- and any other Nodes associated with them -- from being GCed
GC will be slower for NodeSet than currently

which leads me to conclude that option (1) is the better choice at this time.
And a Crazy Idea
However, there is a third option which I'd like to explore:

rewrite the NodeSet class to be a subclass of Array, and to be independent of any specific Document

which would greatly simplify the implementation, but also bring with it the disadvantages of option (2) above.
Next Steps
I'm going to implement (1), but also create a separate issue to drive exploration of (3).

I've opened an issue for exploration of the crazy idea at #2184. I'll work on a less risky fix for this bug this week.

Blurgh, option (1) above breaks the semantics of the NodeSet in that the duplicate we'd create would be unparented from the original document -- which is not going to make any sense for users who end up using NodeSets to collect the results of multiple queries across documents.
I think we need to implement option (2) and accept a slightly slower mark phase. On the plus side, this is exactly what NodeSet-as-subclass-of-Array would do if we decide to eventually reimplement the class.

PR created at #2186.

I'm probably overlooking a few things, but here's what I think is a minimal approach.

The Document object keeps a reference to every Node object in a "node cache", even if the node has been unparented from the DOM tree. Every Node object keeps a reference back to its parent Document in the @document attribute.

There's no need for a node cache or to create these Ruby references. When marking a node it should be enough to mark the Ruby objects for the node's parent and doc which can be looked up using the xmlNode struct. Marking all parents will ultimately mark the document or the root node of "unparented" trees. If an unreferenced Ruby object is freed, this means that its whole tree will be freed as well. The only complication I see is that nodes can be freed in random order.

ensure that NodeSets mark their contents, similar to Array and other standard Ruby container classes.

This seems like the best solution to me.

@nwellnhof Totally agree, right now the codebase reflects our memory management approach from ~2008 when we didn't have a good grasp of libxml2's behaviors.
I'd like to implement something like what you're describing, and that the libxml-ruby gem introduced in https://github.com/xml4r/libxml-ruby/commit/c41f5776e845149e3fbb20c1e10bfd1c30133354. Also see my notes in https://github.com/sparklemotion/nokogiri/issues/2822.
Just a matter of finding the time to do the work.


Fix remove columns from lazy dict

This was introduced in https://github.com/huggingface/datasets/pull/5252 and causing the transformers CI to break: https://app.circleci.com/pipelines/github/huggingface/transformers/53886/workflows/522faf2e-a053-454c-94f8-a617fde33393/jobs/648597
Basically this code should return a dataset with only one column:
from datasets import *

ds = Dataset.from_dict({"a": range(5)})

def f(x):
    x["b"] = x["a"]
    return x

ds = ds.map(f, remove_columns=["a"])

assert ds.column_names == ["b"]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Function to Sum Two Numbers in C

Function to Sum Two Numbers in C

Merged

This is an automated message from Fork, Commit, Merge [BOT].

Thank you for your contribution! Your pull request has been merged. The files have been reset for the next contributor.
What's next?
If you're looking for more ways to contribute, I invite you to check out my other projects. Just click here to find more. These projects contain real issues that you can help resolve. You can also check out the Influences section in the README to find more projects similar to this one.
Also please leave a star to this project if you feel it helped you, i would really appreciate it.
I look forward to seeing your contributions!


Please support mappings with Meta-Key

At the moment vim doesn't really support mapping the Meta (or Alt) key, since it breaks certain characters with diacretics.
Examples for this problem can be seen in the Issue list for vim-rsi or in this issue for vimacs.
However, neovim does support such mappings as stated in their documentation:
Input:
  ALT (|META|) chords always work (even in the |TUI|). Map |<M-| with any key:
  <M-1>, <M-BS>, <M-Del>, <M-Ins>, <M-/>, <M-\>, <M-Space>, <M-Enter>, etc.
  Case-sensitive: <M-a> and <M-A> are two different keycodes.

  ALT in insert-mode behaves like <Esc> if not mapped. |i_ALT|

And that really works. So it obviously isn't impossible to support this.
As it is now this is a problem that is not solvable by any plugins (and instead breaks some plugins like the above mentioned ones) and needs to be resolved in vim internally.
Therefore please support such Meta-mappings like neovim does.

I just tried, it does not work.  Perhaps it works in some places, but not in general.

I suppose not all terminals are supported yet. Technically, this comes from Paul Evan @leonerd libtermkey library. However he urges new projects to not use it anymore but use the libticket library instead. Perhaps he can say how well it is supposed across the various terminals.

libtickit currently uses libtermkey as a dependency; the reason I suggest not to use libtermkey directly is because the entire codebase needs to be copypasted in and just merged together. The two shouldn't be two separate libraries as they really need to integrate far more tightly together than that.

Though just for META the extended encoding in libtermkey shouldn't really be needed, right? It should work in any terminal that uses ESC for meta rather than high-bit encoding (for instance, so that <M-ä> can be encoded as ^[ä, while setting the high bit of already UTF-8 char doesn't make sense).

Well there's always been two parts to this problem: a) that vim doesn't recognise the bytes as they arrive from the TTY, and b) that vim's internals conflate the idea of Alt/Meta prefix and 8bit-high characters. The latter can be seen via e.g. GTK gvim, which doesn't involve a terminal at all. Any use of libtermkey or libtickit would help in the direction of part a, but until part b is solved that's only half a solution.

that vim's internals conflate the idea of Alt/Meta prefix and 8bit-high characters.

This has been improved recently (some weeks/months ago), as can be seen by <M-ä> being properly highlighted in map output at least, which it wasn't previously. But I haven't tested if it works concretely say in gtk gvim.

My usual test is the somewhat-coincidental pairing of Alt-2 vs. ² (being U+00B2, which vim thinks is '2' | 0x80 and hence collides with Alt-2). I've never been able to :map  in vim without it also catching ². If you can map and distinguish the two entirely independently in, say, GTK-based gvim, then that suggests part b is now solved, and can move on to part a.

Paul Evans wrote:

My usual test is the somewhat-coincidental pairing of Alt-2 vs. ²
(being U+00B2, which vim thinks is '2' | 0x80 and hence collides with
Alt-2). I've never been able to :map  in vim without it also
catching ². If you can map and distinguish the two entirely
independently in, say, GTK-based gvim, then that suggests part b is
now solved, and can move on to part a.

We were discussing the claim that Neovim can use Meta mappings, see the
start of this issue.  However, I have not been able to make it work in a
regular xterm.  We first need to see Neovim actually having this work
before I can look into how Vim can support this.
There is one suggestion to make the terminal prefix Esc to indicate
Meta, but that is a terrible idea, since you can type that as well, and
typing Esc is quite common, thus this leads to tricky timeout settings
and failures when replaying a log.  I'm not aware of a reliable
mechanism for the terminal to indicate Meta was pressed.
--
DENNIS:  Listen -- strange women lying in ponds distributing swords is no
basis for a system of government.  Supreme executive power derives
from a mandate from the masses, not from some farcical aquatic
ceremony.
The Quest for the Holy Grail (Monty Python)
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

Yes; at present even libtermkey uses a (short) timer to try to distinguish ESC-prefixed <Alt-letter> sequences, vs the two separate keypresses of <Escape> letter. It's not ideal but it works well enough in practice for most terminals and situations.
A far nicer solution that doesn't at all rely on timing is to use my CSI u encoding scheme. This is the scheme that several terminals now emit, either always or according to an option, and which libtermkey will automatically recognise on receipt.
In this scheme, probably the simplest setup is just to have the <Escape> key encoded as a CSI 27 u seeing as ESC is 27 decimal, and thus they keypress now sends a distinct sequence. The only use of a plain ESC byte on the wire is now part of a longer sequence so you can always wait for it. Alternatively, you could instead keep <Escape> key sending an ESC byte and use CSI u encoding for any <Alt-letter> chorded key. You could even send both at once for extra clarity.

However, I have not been able to make it work in a regular xterm.

The way I make this work is to have the following xrdb settings:
*VT100*eightBitControl: false
*VT100*eightBitInput:   false
*VT100*eightBitOutput:  true

at which point the keys <Alt-2> vs ² send bytes 0x1b 0x32 (ESC 2) vs 0xc2 0xb2 (UTF-8 encoding of U+00B2). At this point, nvim can immediately distinguish the two, and my :imap <Alt-2> ^O:b 2<CR> works just fine while still allowing me to type a ² symbol as regular text.

Paul Evans wrote:

Yes; at present even libtermkey uses a (short) timer to try to
distinguish ESC-prefixed <Alt-letter> sequences, vs the two separate
keypresses of <Escape> letter. It's not ideal but it works well
enough in practice for most terminals and situations.
A far nicer solution that doesn't at all rely on timing is to use my
CSI u encoding scheme. This is the scheme that several terminals now
emit, either always or according to an option, and which libtermkey
will automatically recognise on receipt.
In this scheme, probably the simplest setup is just to have the
<Escape> key encoded as a CSI 27 u seeing as ESC is 27 decimal,
and thus they keypress now sends a distinct sequence. The only use of
a plain ESC byte on the wire is now part of a longer sequence so you
can always wait for it. Alternatively, you could instead keep
<Escape> key sending an ESC byte and use CSI u encoding for any
<Alt-letter> chorded key. You could even send both at once for extra
clarity.

That sounds limited to only Meta/Alt.  How about the combinations with
Ctrl and Shift?
Xterm does encode the modifiers when using function keys.  Vim can
handle that properly.  What we would need is a prefix for normal
characters to pass the modifiers.  It also makes a difference between
Meta and Alt.  The table is:
  Code     Modifiers
---------+---------------------------
   2     | Shift
   3     | Alt
   4     | Shift + Alt
   5     | Control
   6     | Shift + Control
   7     | Alt + Control
   8     | Shift + Alt + Control
   9     | Meta
   10    | Meta + Shift
   11    | Meta + Alt
   12    | Meta + Alt + Shift
   13    | Meta + Ctrl
   14    | Meta + Ctrl + Shift
   15    | Meta + Ctrl + Alt
   16    | Meta + Ctrl + Alt + Shift
---------+---------------------------

--
Courtroom Quote #19:
Q:  Doctor, how many autopsies have you performed on dead people?
A:  All my autopsies have been performed on dead people.
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

However, I have not been able to make it work in a regular xterm.

The way I make this work is to have the following xrdb settings:
*VT100*eightBitControl: false
*VT100*eightBitInput:   false
*VT100*eightBitOutput:  true

at which point the keys <Alt-2> vs ² send bytes 0x1b 0x32 (ESC 2)
vs 0xc2 0xb2 (UTF-8 encoding of U+00B2). At this point, nvim can
immediately distinguish the two, and my :imap <Alt-2> ^O:b 2<CR>
works just fine while still allowing me to type a ² symbol as regular
text.

As far as I can see this is the same as having metaSendsEscape on, using
the Meta/Alt key prefixes the key with an Esc.  With the timing problems
already mentioned.
--
Q: How does a UNIX Guru pick up a girl?
A: look; grep; which; eval; nice; uname; talk; date;
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

That sounds limited to only Meta/Alt. How about the combinations with Ctrl and Shift? Xterm does encode the modifiers when using function keys. Vim can handle that properly. What we would need is a prefix for normal characters to pass the modifiers. It also makes a difference between Meta and Alt. The table is:

Oh, CSI u is a two-position encoding that includes the modifiers, exactly from that table there. So for instance <Ctrl-Shift-Escape> would encode as CSI 27; 6u. I just omitted the second parameter on my earlier example because it would default to 1, being unmodified. This is in fact how most modified keys are already encoded.

As far as I can see this is the same as having metaSendsEscape on, using the Meta/Alt key prefixes the key with an Esc. With the timing problems already mentioned.

Yes, indeed. Without the perfectly unambiguous support of CSI u we're left with this annoying tradeoff. Either accept that you have to use timing information, or accept that you can't distinguish Alt- combinations from Unicode. As someone who types ² quite a lot (e.g. when talking about electronic I²C buses) I found I preferred to make the tradeoff in that direction.
In practice it basically never causes a problem. I set the timeout to 20msec, which is fast enough to be "instant" to human percpection, so pressing a plain <Escape> key feels instant. It's a short enough time that it's basically impossible to accidentally type <Escape> : w say, and have it interpreted as <Alt-:> w, but still being nonzero it covers those occasional cases e.g. where the two bytes of the keypress end up being sent in separate TCP packets that have a short delay over wifi or mobile data.
I accept that it is not perfect, but in my experience (and I've been running this configuration the past 5 years now) it is by far the closest to perfection, at least until universal CSI u support becomes available anywhere.
Plus, application programs can help this situation by accepting CSI u encoding whenever they see it, thus they'll be ready for when more terminals send it.

Paul Evans wrote:

That sounds limited to only Meta/Alt. How about the combinations
with Ctrl and Shift? Xterm does encode the modifiers when using
function keys. Vim can handle that properly. What we would need is a
prefix for normal characters to pass the modifiers. It also makes a
difference between Meta and Alt. The table is:

Oh, CSI u is a two-position encoding that includes the modifiers,
exactly from that table there. So for instance <Ctrl-Shift-Escape>
would encode as CSI 27; 6u. I just omitted the second parameter on
my earlier example because it would default to 1, being unmodified.
This is in fact how most modified keys are already encoded.

Then how do you encode only the Ctrl modifier with some non-ASCII
character?  Since Meta/Alt isn't pressed there would be no Esc to send.
--
Q:   How many hardware engineers does it take to change a lightbulb?
A:   None.  We'll fix it in software.
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

Then how do you encode only the Ctrl modifier with some non-ASCII character? Since Meta/Alt isn't pressed there would be no Esc to send.

CSI u is for any sort of modifier+unicode character. E.g.<Ctrl-é> could be encoded as CSI 233;5 u - 233 is the decimal value of 0xE9, being the codepoint number for é; and 5 is the just-ctrl modifier. We're not "sending an Esc" because of Meta/Alt; we're just sending the CSI encoding.

Paul Evans wrote:

Then how do you encode only the Ctrl modifier with some non-ASCII
character? Since Meta/Alt isn't pressed there would be no Esc to send.

CSI u is for any sort of modifier+unicode character. E.g.<Ctrl-é>
could be encoded as CSI 233;5 u - 233 is the decimal value of 0xE9,
being the codepoint number for é; and 5 is the just-ctrl modifier.
We're not "sending an Esc" because of Meta/Alt; we're just sending the
CSI encoding.

In this case, does the é character also get sent by itself, or does this
mode mean that the terminal only sends this CSI .. u code for all
characters typed?  Or only characters with a modifier that can't be
applied to the base character?
What terminals support this, and what code is used to enable this mode?
And how to disable it again later?
I guess I'm asking for the documentation that details this mode.

/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

In this case, does the é character also get sent by itself, or does this mode mean that the terminal only sends this CSI .. u code for all characters typed? Or only characters with a modifier that can't be applied to the base character?

Yes; it's an alternative to sending the UTF-8 bytes.
The basic idea is that on any keypress of a printing (i.e. Unicode, not cursor/arrow/function key) character, you'll either send UTF-8 bytes or a CSI u encoding, except for the simple C0 single byte values that a lot of code will be expecting - 0x03 for <Ctrl-C> and so on.
The choice is based on the modifiers involved. Ignore shift as it was only used to create the particular Unicode symbol - A vs a, ! vs 1, etc. Of the remaining: If there are no modifiers, send UTF-8. If we can send it as a "legacy" C0, send a C0. Otherwise send CSI u.

What terminals support this

A good question. Anything libvterm-based (i.e. pangoterm plus the :term mode of vim and nvim). xterm can be asked to but last time I looked it was either way too aggressive (i.e. broke all the C0 codes by sending even those as CSI u) or nowhere near enough (i.e. far too many ambiguities).
Possibly terminal-wg may know more on that as they might have a survey of other terminals - since I just run pangoterm I don't really have a good survey of others.

and what code is used to enable this mode? And how to disable it again later?

So far I haven't really found it necessary to need switching on or off. The decision to send a CSI u should only be made for keys that couldn't have been represented without it anyway; for anything else you can just send "legacy" C0s or ESC-prefixing with timing recovery for those - again see above on my "good enough" comments about timing. I guess if you wanted to enable it for all Alt- presses (even those which could be ESC-prefixable e.g. <Alt-a>) or the <Escape> key itself you'd want a mode number for it.
I think terminal-wg again probably have a better overview on the mode number for that.

I guess I'm asking for the documentation that details this mode.

Most folks tend to refer back to my original page I wrote about it many years ago, at
http://www.leonerd.org.uk/hacks/fixterms/
That ends with a note about using S8C1T mode as way to avoid timing information, though that doesn't quite cover the case of wanting <Alt-letter> to still keep its "legacy" encoding and also be able to recover <Escape>. A better solution there is likely to allocate a new mode number for it so as to switch it on in either or both of those cases.

Paul Evans wrote:

In this case, does the é character also get sent by itself, or does
this mode mean that the terminal only sends this CSI .. u code for
all characters typed? Or only characters with a modifier that can't
be applied to the base character?

Yes; it's an alternative to sending the UTF-8 bytes.
The basic idea is that on any keypress of a printing (i.e. Unicode,
not cursor/arrow/function key) character, you'll either send UTF-8
bytes or a CSI u encoding, except for the simple C0 single byte
values that a lot of code will be expecting - 0x03 for <Ctrl-C> and
so on.
The choice is based on the modifiers involved. Ignore shift as it was
only used to create the particular Unicode symbol - A vs a, ! vs 1,
etc. Of the remaining: If there are no modifiers, send UTF-8. If we
can send it as a "legacy" C0, send a C0. Otherwise send CSI u.

After some digging I found that Xterm supports something similar, if the
modifyOtherKeys resource is set (which can be done with an escape
sequence).  But it's not exactly the same.  Xterm did support this
before libvterm.
Unfortunately, it's very hard to support something that is not properly
documented.  So far I haven't found documentation that is explaining
exaclty what key produces what character sequence.  And what is needed
to enable it (and disable it again, e.g. when executing a shell
command).

What terminals support this

A good question. Anything libvterm-based (i.e. pangoterm plus the
:term mode of vim and nvim). xterm can be asked to but last time I
looked it was either way too aggressive (i.e. broke all the C0 codes
by sending even those as CSI u) or nowhere near enough (i.e. far too
many ambiguities).

That's not a widespread use.  Xterm is used in many more places
(including terminals that simulate xterm), thus I would implenent that
first, if it actually works.  I'm sure other terminal implementations
will then follow.

Possibly terminal-wg may know more on that as they might have a survey
of other terminals - since I just run pangoterm I don't really have a
good survey of others.

and what code is used to enable this mode? And how to disable it
again later?

So far I haven't really found it necessary to need switching on or
off. The decision to send a CSI u should only be made for keys that
couldn't have been represented without it anyway; for anything else
you can just send "legacy" C0s or ESC-prefixing with timing recovery
for those - again see above on my "good enough" comments about timing.
I guess if you wanted to enable it for all Alt- presses (even those
which could be ESC-prefixable e.g. <Alt-a>) or the <Escape> key
itself you'd want a mode number for it.

I first need to see a specification of how it works exactly, and see how
this fits in with the (very complicated) support of Vim for mappings and
termcodes.

I think terminal-wg again probably have a better overview on the mode
number for that.

I guess I'm asking for the documentation that details this mode.

Most folks tend to refer back to my original page I wrote about it
many years ago, at
http://www.leonerd.org.uk/hacks/fixterms/

That reads like it's written by a teenager who thinks he knows
everything...  It contains some good ideas, but perhaps you can rephrase
to be more constructive.

That ends with a note about using S8C1T mode as way to avoid timing
information, though that doesn't quite cover the case of wanting
<Alt-letter> to still keep its "legacy" encoding and also be able to
recover <Escape>. A better solution there is likely to allocate a
new mode number for it so as to switch it on in either or both of
those cases.

What is missing is how this can be done in a backwards-compatible way.
And there is no mention of the mechanism that xterm already implements.
--
There are three kinds of people: Those who can count & those who can't.
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

After some digging I found that Xterm supports something similar, if the modifyOtherKeys resource is set (which can be done with an escape sequence). But it's not exactly the same. Xterm did support this before libvterm. Unfortunately, it's very hard to support something that is not properly documented. So far I haven't found documentation that is explaining exaclty what key produces what character sequence. And what is needed to enable it (and disable it again, e.g. when executing a shell command).

Yes - modifyOtherKeys is a precedent here. Only xterm's behaviour is annoyingly not quite right in whichever mode you set it in. Either it doesn't decide to use CSI u often enough to distinguish keys, or it uses it too much and breaks the legacy C0 encodings so things like termios get confused. The behaviour we came up with is a middle-ground here that distinguishes keys without breaking that.

I first need to see a specification of how it works exactly, and see how this fits in with the (very complicated) support of Vim for mappings and termcodes.

Yes indeed - working on that now.

 It contains some good ideas, but perhaps you can rephrase to be more constructive.

Yah - I think it probably was ;)
I've been working on rewording it and turning in into an actual spec.
It currently lives here:
https://docs.google.com/document/d/1Ywq9eVHlGhpGodG7sSd1EbxsTeqJptlhCrYmMSyOn8E
Though that's rather work-in-progress at the moment. I'll update when I have something polished.

What is missing is how this can be done in a backwards-compatible way. And there is no mention of the mechanism that xterm already implements.

Well as I said - CSI u is what xterm already does.

see also https://groups.google.com/forum/#!topic/vim_dev/4fwJQtsCSJI

Thomas Dickey is working on better documentation.  Although the modifyOtherKeys setting isn't ideal, it at least is wide spread enough that many users can take advantage of it.  Vim can deal with the drawbacks (e.g. falling back to the old key if there is no mapping for the key + modifier).
I had to modify libvterm (the copy included with Vim) to do the same. Vim can also recognize the CSI u form, but libvterm never puts Meta/Alt as a modifier and prepends Esc instead.  Thus using the xterm encoding works better for Vim.
I checked with a few mappings, and at least ,  and  work.   is intercepted by the system (triggers a menu?).
It doesn't work yet for mapping   and  separately.  Well, they are separate but exactly the wrong way around.  I'll see if that can be fixed without breaking something.

Hmm, not sure if I misunderstood that announcement.
I just tried with vim 8.1.2135.
I have tpope/vim-rsi installed which maps <m-d> in insert mode to <c-o>dw. In previous versions this conflicts with the german character ä. So if I enter ä it would delete up to the next word instead.
This still happens with that new version and even displaying the mapping reveals that conflict:
:imap <m-d>
i  ä           * <C-O>dw

Is that keybinding not affected by the above mentioned changes? Sorry, if my question may look dumb, but I am not very firm in the details of such terminal behavior.

I believe you need to use xterm and have the modifyOtherKeys  setting set.

Marco Herrn wrote:

Hmm, not sure if I misunderstood that announcement.
I just tried with vim 8.1.2135.
I have tpope/vim-rsi installed which maps <m-d> in insert mode to <c-o>dw. In previous versions this conflicts with the german character ä. So if I enter ä it would delete up to the next word instead.
This still happens with that new version and even displaying the mapping reveals that conflict:
:imap <m-d>
i  ä           * <C-O>dw

Is that keybinding not affected by the above mentioned changes? Sorry,
if my question may look dumb, but I am not very firm in the details of
such terminal behavior.

First check that you actually enabled the codes by typing CTRL-V and
then the key.  You should get something with [27;.
I still have to look at how mappings are used.  My keyboard doesn't
produce a key like ä, thus it's a bit difficult for me to try it out.
I do also see the problem with  and , which can now be
recognized as two different keys, but mapping doesn't work yet.
You should at least be able to map the literal sequence:
:inoremap CTRL-V {the key with meta}  <C-O>dw

--
There is no right or wrong, there is only your personal opinion.
(Bram Moolenaar)
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

First check that you actually enabled the codes by typing CTRL-V and
then the key.  You should get something with [27;.

Yes, that does work. I get ^[[27;3;100~ when entering <m-d> and ä when entering ä.
And it does that only in xterm. In urxvt I get ^[d when entering <m-d>. So I assume what Christian was writing above is correct that this only works with xterm and modifyOtherKeys enabled.

You should at least be able to map the literal sequence:
:inoremap CTRL-V {the key with meta}  dw

That works too. I can then use <m-d> to delete up to the next word and also enter the character ä just by entering it without having them interfer with each other.
But I noticed that this also works with older versions of vim (in my case the version shipped with Debian Buster: 8.1.0875). So this has nothing to do with the new changes?

My keyboard doesn't
produce a key like ä, thus it's a bit difficult for me to try it out.

Even though I am using a german keyboard, I mainly use an english keymap and therefore use the compose key to enter the german umlauts. For example to get an ä I use <compose>"a.  If you have a compose key enabled, maybe that's a way for you to enter these characters more easily.

First check that you actually enabled the codes by typing CTRL-V and
then the key.  You should get something with [27;.

Yes, that does work. I get ^[[27;3;100~ when entering <m-d> and
ä when entering ä.
And it does that only in xterm. In urxvt I get ^[d when entering
<m-d>. So I assume what Christian was writing above is correct that
this only works with xterm and modifyOtherKeys enabled.

Yes, other terminals prepend an Esc when using the Alt key, which is
confusing since you can also type "Esc d".  Tricky timing is required to
guess what the sequence means.

You should at least be able to map the literal sequence:
:inoremap CTRL-V {the key with meta}  dw

That works too. I can then use <m-d> to delete up to the next word
and also enter the character ä just by entering it without having
them interfer with each other.  But I noticed that this also works
with older versions of vim (in my case the version shipped with Debian
Buster: 8.1.0875). So this has nothing to do with the new changes?

I need to fix the mapping mechanism.  Vim now understands the Esc
sequence and applies the Meta/Alt key if it wasn't used in a mapping.

My keyboard doesn't produce a key like ä, thus it's a bit difficult
for me to try it out.

Even though I am using a german keyboard, I mainly use an english
keymap and therefore use the compose key to enter the german umlauts.
For example to get an ä I use <compose>"a.  If you have a compose
key enabled, maybe that's a way for you to enter these characters more
easily.

I don't have a compose key...
--
If an elephant is left tied to a parking meter, the parking fee has to be paid
just as it would for a vehicle.
[real standing law in Florida, United States of America]
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

This should work now as of patch 8.1.2146.  Add this to your .vimrc:
" Enable modifyOtherKeys
let &t_TI = "<Esc>[>4;2m"
let &t_TE = "<Esc>[>4;m"
@leonerd  please look at the simple changes I made to libvterm to support the xterm modifyOtherKeys.  It doesn't change any existing behavior and makes it work identical to xterm.
This is part of patch 8.1.2134.
Let me know if this causes trouble or something still doesn't work.

@leonerd please look at the simple changes I made to libvterm to support the xterm modifyOtherKeys. It doesn't change any existing behavior and makes it work identical to xterm.
This is part of patch 8.1.2134.

@brammool - Hi. Can you point me in the direction of that patch - is it available somewhere easily?
I think more generally we might want to do a bit of a resync between your fork of libvterm and the main upstream one; doubly so now that I have an actual numbered version release of it. I've been working with the neovim folks to work on some new features and fix a few other bits, so it would be nice to get everyone back in sync again rather than have vim continue down the direction of a fork.

Is this what you are looking for?
https://github.com/vim/vim/commit/6a0299d8f4c7a64c64d60a6bb39cfe6eaf892247
or are you more looking for a diff between your upstream libvterm and the one bundled with Vim?

Ah, in the first instance that looks fine for this particular libvterm change, so I can see that.
But for the more general part I think we'd just want to diff between the two sides overall, and see what comes up.

But for the more general part I think we'd just want to diff between the two sides overall, and see what comes up.

Okay, I just cloned neovims/libvterm and replaced it with vims fork:
https://github.com/neovim/libvterm/commit/9ddd0d8a28a81569015c04f17de695c81f8824c1

to the ones reading this from the ml:
I don't know why initially this was shown as a commit in neovims repository. The commit is in my private fork, I just fixed the commit link.

Since libvterm is now in more distributions (due to neovim), it would be especially good to consolidate the differences so Vim can stop embedding libvterm and just use the system library.

There's no libvterm built in to macOS.

There's no libvterm built in to macOS.

Neither are a lot of libraries that one needs to build software on macOS.  That's usually where Homebrew comes in, or building the libraries yourself.

Neither are a lot of libraries that one needs to build software on macOS

All the ones needed to build a good Vim are, though.

That's usually where Homebrew comes in, or building the libraries yourself.

...or bundling this tiny thing along with Vim, so nobody has to take those PITA steps.  Something as basic and fundamental as Vim should just build out of the box.

Paul Evans wrote:

@leonerd please look at the simple changes I made to libvterm to
support the xterm modifyOtherKeys. It doesn't change any existing
behavior and makes it work identical to xterm.
This is part of patch 8.1.2134.

@brammool - Hi. Can you point me in the direction of that patch - is
it available somewhere easily?

https://groups.google.com/forum/#!searchin/vim_dev/8.1.2134|sort:date/vim_dev/xcpXAoWVml0/Tqki2VeWCgAJ

I think more generally we might want to do a bit of a resync between
your fork of libvterm and the main upstream one; doubly so now that I
have an actual numbered version release of it. I've been working with
the neovim folks to work on some new features and fix a few other
bits, so it would be nice to get everyone back in sync again rather
than have vim continue down the direction of a fork.

Indeed.  I recently renamed the files back to the original name to make
merging a bit easier.
What version of libvterm should we merge from?  I know about the neovim
copy on github: https://github.com/neovim/libvterm
I'm not sure how this relates to yours.
--
Why is "abbreviation" such a long word?
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

Since libvterm is now in more distributions (due to neovim), it would
be especially good to consolidate the differences so Vim can stop
embedding libvterm and just use the system library.

That's very unlikely to happen, since Vim has some Vim-specific changes.
And I tend to add features once in a while, would not want to wait for
system library updates for Vim to be able to use them.  And on
MS-Windows one would have to download and upack it separately, not an
attractive option.
--
ARTHUR: CHARGE!
[The mighty ARMY charges.  Thundering noise of feet.  Clatter of coconuts.
Shouts etc.   Suddenly there is a wail of a siren and a couple of police
cars roar round in front of the charging ARMY and the POLICE leap out and
stop them.  TWO POLICEMAN and the HISTORIAN'S WIFE.  Black Marias skid up
behind them.]
HISTORIAN'S WIFE: They're the ones, I'm sure.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

@brammool wrote...

This should work now as of patch 8.1.2146. Add this to your .vimrc:
" Enable modifyOtherKeys
let &t_TI = "[>4;2m"
let &t_TE = "[>4;m"
@leonerd please look at the simple changes I made to libvterm to support the xterm modifyOtherKeys. It doesn't change any existing behavior and makes it work identical to xterm.
This is part of patch 8.1.2134.
Let me know if this causes trouble or something still doesn't work.

After upgrading to  Vim-8.2 this patch has caused issues with me...
I use "autocmd" to read in a binary encrypted file, which is then decrypted using a password read directly from the users TTY.
If I type non-shifted letters... the password for decryption is fine.  But if I say use 'A' That is a capital-A,  the password being entered instead of getting a 'A' gets '\e[27;2;65~'  which is the ANSI code for a Shifted A.   As you can imagine the command fails to decrypt the file being edited, if the password had any shifted characters in it!
If I use a :%!command  to do the decryption there is no problem...  The input is read right, so VIM is disabling it for that.   It only becomes a problem when I do this from a "autocmd"
I am currently disabling this by sending a the key sequence  '\e[>4n'  to the terminal, before trying to read the password.  Of course this only works if I have wrapper I can program around the de/encrypt command.

I use "autocmd" to read in a binary encrypted file, which is then decrypted using a password read directly from the users TTY.

Can you show the actual setting?

@k-takata See the bug report I just made for an example

The solution (https://github.com/vim/vim/issues/4974#issuecomment-541431801) talked in this thread doesn't apply to Gvim, does it? Is the differentiation of meta/alt+key from accented letters still impossible there? I.e <M-v> seems to still mean ö on my Gvim...

Indeed so. The trouble overall is that vim's internals are built presuming a terminal-encoding view of the world. Even from GUI systems that should not need to care about terminal encodings, the vim core still conflates the Alt- modified with ESC prefixing (as well as other confusions such as Ctrl-I vs Tab). All the discussions above about CSIu encodings and so on are quite a distraction from that central core problem.
One of the main changes in nvim's core was to migrate to a representation of keypresses, distinct from the raw terminal bytes that might have been received in order to express them. With that change in place, it becomes possible to think about the concept of Alt- modifiers separate from ESC prefixing, or Ctrl-I separate from Tab. Once you do that, any GUI system which doesn't suffer terminal's conflation problems is now fixed. The remaining questions of how you'd represent that distinction on a terminal specifically become a separate question.
But the first step you have to make is to allow the input queue system to conceive of the idea of Alt- modify vs ESC prefix.

This remarks appears to be out of context.  There is no Esc prefixing in the GUI.  Most likely the input method converts  to a character before Vim even sees it.  gVim is perfectly able to map  separately from entering that character with some input method.

Ah, actually Andri was correct, then handling of modifiers was changed in the terminal for modifyOtherKeys, but this was not changed for the GUI.  I think we can make that change now, but I'm worried that it might break for some users...  Making the change might be the only way to find out.


Fixed #27133 -- clarify loading data fixtures

See ticket: https://code.djangoproject.com/ticket/27133

merged in eaa6ea2f37509f2e54de5268f32469deeff89eb9, thanks!


Pad mask with zeros for non-square attention matrices

The current implementation of the mask assumes the attention matrix is square, which is true if there is no cache. However, if one wishes to produce multiple tokens at a time, such as in speculative decoding implementations, a rectangular mask is necessary.
This change pads the bottom of the mask with zeros so multi-token decoding with a cache works correctly.

@kevmo314 are you coming back to this?

@kevmo314 are you coming back to this?

Yes I will come back to this shortly.

@awni I've updated the implementation!

Looks like the test failed? Could you take a look?


🐛 [Bug] GDT Transaction ID is missing

🐛 Bugreport

GDT Transaction ID is missing
We get null values for GDT transaction ids (or so I understood). This made us switch from a direct GDT-Connection to the Community-Server-Tunnel (why? @Mogge ).
Apparently we get null values for the transaction ids, which are used as unique identifier for a GDT-Transaction analog to the GDD-Transactions

Please check again.

I think @Mogge switched to Community-Server-Tunnel because the gdt call needs instead of session_id email as parameter.


Chore/Include the installation instruction on readme

This PR will allow the npm package users to be able to get the instruction on how to install this package, This is not currently provided on the README.md file.

Done in main.


altered versus abnormal susceptibility or response terms

The MP currently uses both "altered" and "abnormal" in the primary term label of various existing susceptibility/response/tumor related etc terms
Examples using "altered":
altered response to CNS ischemic injury MP:0003075
altered response of heart to induced stress MP:0004484
altered tumor pathology MP:0010639
altered tumor susceptibility MP:0002166
altered metastatic potential MP:0000858
altered susceptibility to infection MP:0001793
altered position of colliculi MP:0000901
Examples using "abnormal":
abnormal susceptibility to weight gain MP:0011117
abnormal susceptibility to induced colitis MP:0013296
abnormal susceptibility to aortic cartilaginous metaplasia MP:0020092
abnormal response to vitamins MP:0003639
abnormal response to infection MP:0005025
abnormal physiological response to xenobiotic MP:0008872
Consider standardizing using only "abnormal" in the primary term label, making "altered" an exact synonym???

Should be consistent and use abnormal make altered an exact synonym

Primary term label has been changed to "abnormal".
Added "altered" as an exact synonym only to those terms that previously used "altered" in the term label.


[BUG] dns resolution fails as soon as an edge node is connected

What happened:
As soon as i add an edgenode to the yurtcluster the dns resolution works only sporadically. This is due to the fact that the coredns service addresses the coredns on the egdenode. The cluster is currently running without Raven.
The following output from within a pod running on master node:
root@nginx-75bcf88584-vzdjv:/# ping google.com
PING google.com (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=57 time=0.557 ms
64 bytes from fra16s53-in-f14.1e100.net (<IP_ADDRESS>): icmp_seq=2 ttl=57 time=0.540 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=57 time=0.524 ms
64 bytes from fra16s53-in-f14.1e100.net (<IP_ADDRESS>): icmp_seq=4 ttl=57 time=0.531 ms
64 bytes from fra16s53-in-f14.1e100.net (<IP_ADDRESS>): icmp_seq=5 ttl=57 time=0.509 ms
64 bytes from fra16s53-in-f14.1e100.net (<IP_ADDRESS>): icmp_seq=6 ttl=57 time=0.557 ms
64 bytes from <IP_ADDRESS>: icmp_seq=7 ttl=57 time=0.590 ms
64 bytes from fra16s53-in-f14.1e100.net (<IP_ADDRESS>): icmp_seq=8 ttl=57 time=0.612 ms
64 bytes from fra16s53-in-f14.1e100.net (<IP_ADDRESS>): icmp_seq=9 ttl=57 time=0.553 ms
C64 bytes from <IP_ADDRESS>: icmp_seq=10 ttl=57 time=0.559 ms
--- google.com ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 36028ms
rtt min/avg/max/mdev = 0.509/0.553/0.612/0.028 ms

The node is in the right node pool:
k get nodes --show-labels
NAME                    STATUS   ROLES                  AGE     VERSION    LABELS
jakku-control-plane-1   Ready    control-plane,master   2d13h   v1.22.17   apps.openyurt.io/desired-nodepool=master,apps.openyurt.io/nodepool=master,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=jakku-control-plane-1,kubernetes.io/os=linux,mynetwork-big=,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=,openyurt.io/is-edge-worker=false,raven.openyurt.io/gateway=gw-cloud
jakku-control-plane-2   Ready    control-plane,master   2d13h   v1.22.17   apps.openyurt.io/desired-nodepool=master,apps.openyurt.io/nodepool=master,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=jakku-control-plane-2,kubernetes.io/os=linux,mynetwork-big=,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=,openyurt.io/is-edge-worker=false,raven.openyurt.io/gateway=gw-cloud
jakku-master            Ready    control-plane,master   2d13h   v1.22.17   apps.openyurt.io/desired-nodepool=master,apps.openyurt.io/nodepool=master,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=jakku-master,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=,openyurt.io/is-edge-worker=false,raven.openyurt.io/gateway=gw-cloud
jellyfish               Ready    <none>                 99m     v1.22.17   apps.openyurt.io/desired-nodepool=edge,apps.openyurt.io/nodepool=edge,beta.kubernetes.io/arch=arm64,beta.kubernetes.io/os=linux,kubernetes.io/arch=arm64,kubernetes.io/hostname=jellyfish,kubernetes.io/os=linux,mynetwork-big=,openyurt.io/is-edge-worker=true,raven.openyurt.io/gateway=gw-edge

k get nodepools.apps.openyurt.io
NAME     TYPE    READYNODES   NOTREADYNODES   AGE
edge     Edge    1            0               70m
master   Cloud   3            0               46h
worker   Cloud   0            0               46h

Kubeproxy feature gate is enabled:
k get cm -n kube-system kube-proxy -o yaml
apiVersion: v1
data:
  config.conf: |-
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    bindAddress: <IP_ADDRESS>
    featureGates: # 1. enable EndpointSliceProxying feature gate.
      EndpointSliceProxying: true
    bindAddressHardFail: false
    clientConnection:
      acceptContentTypes: ""
      burst: 0
      contentType: ""
      qps: 0
    clusterCIDR: <IP_ADDRESS>/16
    configSyncPeriod: 0s
    conntrack:
      maxPerCore: null
      min: null
      tcpCloseWaitTimeout: null
      tcpEstablishedTimeout: null
    detectLocalMode: ""
    enableProfiling: false
    healthzBindAddress: ""
    hostnameOverride: ""
    iptables:
      masqueradeAll: false
      masqueradeBit: null
      minSyncPeriod: 0s
      syncPeriod: 0s
...
kind: ConfigMap
metadata:
 ...
  labels:
    app: kube-proxy
  name: kube-proxy
  namespace: kube-system

kubedns  has the right topology key:
k get service -n kube-system kube-dns -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    openyurt.io/topologyKeys: kubernetes.io/zone
    prometheus.io/port: "9153"
    prometheus.io/scrape: "true"
  creationTimestamp: "2023-04-18T21:54:51Z"
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: CoreDNS
  name: kube-dns
  namespace: kube-system
  resourceVersion: "1271960"
  uid: c3427928-6c13-4020-a3b2-267c4f992d9f
  ...

everything runns without problems:
k get pod -n kube-system
NAME                                            READY   STATUS    RESTARTS        AGE
coredns-4njqd                                   1/1     Running   0               46h
coredns-pp9qq                                   1/1     Running   0               28m
coredns-qqh5n                                   1/1     Running   0               46h
coredns-wstk7                                   1/1     Running   0               46h
etcd-jakku-control-plane-1                      1/1     Running   0               2d13h
etcd-jakku-control-plane-2                      1/1     Running   0               2d13h
etcd-jakku-master                               1/1     Running   0               2d13h
kube-apiserver-jakku-control-plane-1            1/1     Running   1 (2d13h ago)   2d13h
kube-apiserver-jakku-control-plane-2            1/1     Running   0               2d13h
kube-apiserver-jakku-master                     1/1     Running   0               2d13h
kube-controller-manager-jakku-control-plane-1   1/1     Running   1 (42h ago)     2d13h
kube-controller-manager-jakku-control-plane-2   1/1     Running   1 (42h ago)     2d13h
kube-controller-manager-jakku-master            1/1     Running   2 (42h ago)     2d13h
kube-proxy-5zg82                                1/1     Running   0               17m
kube-proxy-8cgj6                                1/1     Running   0               17m
kube-proxy-gdvxp                                1/1     Running   0               17m
kube-proxy-hs572                                1/1     Running   0               17m
kube-scheduler-jakku-control-plane-1            1/1     Running   0               2d13h
kube-scheduler-jakku-control-plane-2            1/1     Running   1 (42h ago)     2d13h
kube-scheduler-jakku-master                     1/1     Running   3 (42h ago)     2d13h
yurt-app-manager-6fd8dcd6b4-d68v7               1/1     Running   4 (24h ago)     46h
yurt-controller-manager-9d9f44d45-c5lz2         1/1     Running   3 (42h ago)     46h
yurt-hub-jellyfish

controller manager has no errors:
k logs -n kube-system yurt-controller-manager-9d9f44d45-c5lz2
I0421 10:21:45.890652       1 servicetopology.go:166] the nodes record of nodepool edge is changed from [] to [jellyfish].
I0421 10:21:45.892531       1 servicetopology.go:193] according to the change of the nodepool edge, enqueue endpoints: [kube-system/kube-dns]
I0421 10:21:45.892757       1 servicetopology.go:201] according to the change of the nodepool edge, enqueue endpointslice: [kube-system/kube-dns-dfnjz]
I0421 11:02:32.186770       1 servicetopology.go:177] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpoints: [kube-system/kube-dns]
I0421 11:02:32.186823       1 servicetopology.go:185] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpointslices: [kube-system/kube-dns-dfnjz]
I0421 11:03:34.471636       1 servicetopology.go:177] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpoints: [kube-system/kube-dns]
I0421 11:03:34.471694       1 servicetopology.go:185] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpointslices: [kube-system/kube-dns-dfnjz]
I0421 11:05:08.107732       1 servicetopology.go:177] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpoints: [kube-system/kube-dns]
I0421 11:05:08.107764       1 servicetopology.go:185] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpointslices: [kube-system/kube-dns-dfnjz]
I0421 11:06:18.906667       1 servicetopology.go:177] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpoints: [kube-system/kube-dns]
I0421 11:06:18.906718       1 servicetopology.go:185] the topology configuration of svc kube-system/kube-dns is changed, enqueue endpointslices: [kube-system/kube-dns-dfnjz]

Do you have any ideas?
Thank!
Environment:

OpenYurt version: 1.2
Kubernetes version (use kubectl version): v1.22.17

/kind bug

@batthebee Thanks for raising the issue about dns resolution problem. it seems that service topology is not worked for the dns service on edge node.  please check the yurthub cache data in /etc/kubernetes/cache/kube-proxy/endpointslices/kube-system/kube-dns file whether endpoints are filtered or not?

I'm in control of the device in the exact same cluster. Here is the log of the file you asked for on the jellyfish edge node:
cat /etc/kubernetes/cache/kube-proxy/endpointslices.v1.discovery.k8s.io/kube-system/kube-dns-dfnjz  | jq
{
  "kind": "EndpointSlice",
  "apiVersion": "discovery.k8s.io/v1",
  "metadata": {
    "name": "kube-dns-dfnjz",
    "generateName": "kube-dns-",
    "namespace": "kube-system",
    "uid": "9e711890-7122-4caf-a74e-1564293b9f0b",
    "resourceVersion": "2780720",
    "generation": 92,
    "creationTimestamp": "2023-04-18T21:54:56Z",
    "labels": {
      "endpointslice.kubernetes.io/managed-by": "endpointslice-controller.k8s.io",
      "k8s-app": "kube-dns",
      "kubernetes.io/cluster-service": "true",
      "kubernetes.io/name": "CoreDNS",
      "kubernetes.io/service-name": "kube-dns"
    },
    "annotations": {
      "endpoints.kubernetes.io/last-change-trigger-time": "2023-04-24T05:04:29Z",
      "openyurt.io/update-trigger": "1682080547"
    },
    "ownerReferences": [
      {
        "apiVersion": "v1",
        "kind": "Service",
        "name": "kube-dns",
        "uid": "c3427928-6c13-4020-a3b2-267c4f992d9f",
        "controller": true,
        "blockOwnerDeletion": true
      }
    ],
    "managedFields": [
      {
        "manager": "yurt-controller-manager",
        "operation": "Update",
        "apiVersion": "discovery.k8s.io/v1",
        "time": "2023-04-19T12:49:02Z",
        "fieldsType": "FieldsV1",
        "fieldsV1": {
          "f:metadata": {
            "f:annotations": {
              ".": {},
              "f:openyurt.io/update-trigger": {}
            }
          }
        }
      },
      {
        "manager": "kube-controller-manager",
        "operation": "Update",
        "apiVersion": "discovery.k8s.io/v1",
        "time": "2023-04-21T12:42:45Z",
        "fieldsType": "FieldsV1",
        "fieldsV1": {
          "f:addressType": {},
          "f:endpoints": {},
          "f:metadata": {
            "f:annotations": {
              "f:endpoints.kubernetes.io/last-change-trigger-time": {}
            },
            "f:generateName": {},
            "f:labels": {
              ".": {},
              "f:endpointslice.kubernetes.io/managed-by": {},
              "f:k8s-app": {},
              "f:kubernetes.io/cluster-service": {},
              "f:kubernetes.io/name": {},
              "f:kubernetes.io/service-name": {}
            },
            "f:ownerReferences": {
              ".": {},
              "k:{\"uid\":\"c3427928-6c13-4020-a3b2-267c4f992d9f\"}": {}
            }
          },
          "f:ports": {}
        }
      }
    ]
  },
  "addressType": "IPv4",
  "endpoints": [
    {
      "addresses": [
        "<IP_ADDRESS>"
      ],
      "conditions": {
        "ready": true,
        "serving": true,
        "terminating": false
      },
      "targetRef": {
        "kind": "Pod",
        "namespace": "kube-system",
        "name": "coredns-qflmh",
        "uid": "9e66bef8-24d1-4b8c-a4d9-f17353a8d4d4",
        "resourceVersion": "2780717"
      },
      "nodeName": "jellyfish"
    }
  ],
  "ports": [
    {
      "name": "metrics",
      "protocol": "TCP",
      "port": 9153
    },
    {
      "name": "dns",
      "protocol": "UDP",
      "port": 53
    },
    {
      "name": "dns-tcp",
      "protocol": "TCP",
      "port": 53
    }
  ]
}

Note, that the current openyurt.io/topologyKeys is set to openyurt.io/nodepool.
$ kubectl  get pods -o wide | grep coredns
Found existing global alias for "| grep". You should use: "G"
coredns-dvqz8                                   1/1     Running   0               2d17h   <IP_ADDRESS>       jakku-control-plane-2   <none>           <none>
coredns-qflmh                                   1/1     Running   0               4m24s   <IP_ADDRESS>       jellyfish               <none>           <none>
coredns-qkmcm                                   1/1     Running   0               2d17h   <IP_ADDRESS>       jakku-master            <none>           <none>
coredns-wjm8k                                   1/1     Running   0               2d17h   <IP_ADDRESS>       jakku-control-plane-1   <none>           <none>

**Another example to understand the issue: **
There's an openyurt cluster with three master nodes and one cloud worker node (no edge worker node present).
There are two nodepools: master and worker - each targeting the correct nodes.
There is a YurtAppDaemon with one replica that targets both nodepools: worker and master
There is a service that ends at both pods (for worker and master) and has the annotation openyurt.io/topologyKeys is: openyurt.io/nodepool
So what's the point?

When trying to resolve the service from the worker nodes pod, the worker node will always be addressed. This is what it is supposed to do.
When trying to resolve the service from the master nodes pod, there will be a round robin between the pod that is running on the master node and the other on the worker node. This breaks the desired behavior.  I'd expect, that the master nodes pod would be always addressed.

I'm in control of the device in the exact same cluster. Here is the log of the file you asked for on the jellyfish edge node:
cat /etc/kubernetes/cache/kube-proxy/endpointslices.v1.discovery.k8s.io/kube-system/kube-dns-dfnjz  | jq

...

"addressType": "IPv4",
"endpoints": [
{
"addresses": [
"<IP_ADDRESS>"
],
"conditions": {
"ready": true,
"serving": true,
"terminating": false
},
...
}

@siredmar There are only one endpoint in endpointslice of kube-dns cache, so dns resolution can be worked on jellyfish edge node.
The reason why only one endpoint in kube-dns endpointslice because yurthub can filter the endpointslice according to the service annotation.  and you can dive the code here: https://github.com/openyurtio/openyurt/blob/master/pkg/yurthub/filter/servicetopology/filter.go#L187
so if we want to make service topology function can work, we need to make sure yurthub component run on this node.

So we need also connect the control plane nodes via yurthub to enable the service topology filter mechanism?

@rambohe-ch can you explain what we need to do for the cloud worker nodes? As i understand correctly the cloud mode of yurthub does not do caching. Is the service filter mechanism also enabled for worker nodes? Because it seems not to work either.

@rambohe-ch
Let me try to summarize this:
As long as a node is not connected via yurthub, it will try to resolve services via coredns instances, which may not be reachable, e.g. because they are placed in the edge.
Concretely it can be that my masternode has a pod running, which tries to resolve another service. This name resolution fails from time to time, because it also addresses the coredns on my edge node.
i can only prevent this if i also connect my master node via yurthub or if i make sure that no pod lands on my master node.
Is that correct?
So, now we want to run not only one master node, but a HA cluster consisting of three control-planes.
That means I have to set up three servers, the first one with yurtadm init and the other two with yurtadm join.
However, I don't think there is an option in yurtadm join to join a control-plane node. It is missing something like --control-plane --certificate-key
So far right?
What steps are necessary to set up a HA cluster where the control-plane nodes work correctly? Do I have to set up a "normal" kubeadm cluster first and then convert it node by node? Or is there an easier way?

@batthebee Thanks for your kindly response.

@rambohe-ch
Let me try to summarize this:
As long as a node is not connected via yurthub, it will try to resolve services via coredns instances, which may not be reachable, e.g. because they are placed in the edge.
Concretely it can be that my masternode has a pod running, which tries to resolve another service. This name resolution fails from time to time, because it also addresses the coredns on my edge node.
i can only prevent this if i also connect my master node via yurthub or if i make sure that no pod lands on my master node.
Is that correct?

if you want to use service topology that provided by OpenYurt, you need to make sure kube-proxy connect kube-apiserver via yurthub(if there is yurthub pod on the node, kube-proxy will connect kube-apiserver via yurthub automatically).

So, now we want to run not only one master node, but a HA cluster consisting of three control-planes.
That means I have to set up three servers, the first one with yurtadm init and the other two with yurtadm join. However, I don't think there is an option in yurtadm join to join a control-plane node. It is missing something like --control-plane --certificate-key
So far right?
What steps are necessary to set up a HA cluster where the control-plane nodes work correctly? Do I have to set up a "normal" kubeadm cluster first and then convert it node by node? Or is there an easier way?

sorry, it is not recommended to run yurthub pod on the master node, because it maybe reduce the stability of master node when a new component(yurthub) was placed between kubelet/kube-proxy/etc. and kube-apiserver.
so i suggest that you don't place your component which need to use dns resolution on master node, how about deploy these components on nodes that yurtadm join with cloud mode?


ci: model_replay improvements

comment the new ref + expend on diff

new ref: https://raw.githubusercontent.com/commaai/ci-artifacts/refs/heads/model_replay_mr_improv/2f4452b03ccb98f0|2022-12-03--13-45-30_model_tici_cf94e9a4763c48ef4930b61538d641765151a534.bz2All Model Replay Plots


Error message trying to get into Reports screeen

Defective ToolKit Feature Name: Income Vs. Expense Report
Expected Behavior
I should be able to open up the Income Vs. Expense Report with no error message.  Google Chrome Version 53.0.2785.89 m
Actual Behavior
This is the message that I receive when I try to enter the Income Vs. Expense Report in Google Chrome Version 53.0.2785.89 m.  This started happening in Toolkit Version 0.7.2.    I can't select another report because Toolkit is not letting me.

Steps to Reproduce the Behavior

Open YNAB
Click on Reports
Select Income Vs. Expense Report
Receive said error

Export of Toolkit Settings (Bottom Left of Settings Page)
[{"key":"accountTransactionSearch","value":true},{"key":"accountsDisplayDensity","value":"0"},{"key":"accountsEmphasizedOutflows","value":true},{"key":"accountsSelectedTotal","value":false},{"key":"accountsStripedRows","value":true},{"key":"activityTransactionLink","value":true},{"key":"budgetBalanceToZero","value":true},{"key":"budgetProgressBars","value":"0"},{"key":"budgetQuickSwitch","value":false},{"key":"budgetRowsHeight","value":"0"},{"key":"categoryActivityPopupWidth","value":"0"},{"key":"changeEnterBehavior","value":true},{"key":"checkCreditBalances","value":true},{"key":"checkNumbers","value":false},{"key":"collapseExpandBudgetGroups","value":true},{"key":"collapseSideMenu","value":true},{"key":"colourBlindMode","value":false},{"key":"currentMonthIndicator","value":true},{"key":"daysOfBuffering","value":true},{"key":"daysOfBufferingHistoryLookup","value":"0"},{"key":"editButtonPosition","value":"0"},{"key":"enableRetroCalculator","value":true},{"key":"enlargeCategoriesDropdown","value":true},{"key":"enterInRegisterNow","value":true},{"key":"exportTransactions","value":false},{"key":"goalIndicator","value":false},{"key":"goalIndicatorWarningColor","value":false},{"key":"googleFontsSelector","value":"0"},{"key":"hideAOM","value":false},{"key":"highlightNegativesNegative","value":false},{"key":"importNotification","value":false},{"key":"incomeFromLastMonth","value":"1"},{"key":"l10n","value":"0"},{"key":"largerClickableIcons","value":false},{"key":"monthlyNotesPopupWidth","value":"0"},{"key":"moveMoneyAutocomplete","value":false},{"key":"moveMoneyDialog","value":false},{"key":"pacing","value":true},{"key":"printingImprovements","value":true},{"key":"privacyMode","value":false},{"key":"reconciledTextColor","value":"0"},{"key":"removePositiveHighlight","value":false},{"key":"removeZeroCategories","value":false},{"key":"reports","value":true},{"key":"resizeInspector","value":true},{"key":"rightClickToEdit","value":true},{"key":"runningBalance","value":false},{"key":"seamlessBudgetHeader","value":false},{"key":"showIntercom","value":true},{"key":"splitKeyboardShortcut","value":true},{"key":"splitTransactionAutoAdjust","value":true},{"key":"squareNegativeMode","value":false},{"key":"swapClearedFlagged","value":false},{"key":"toBeBudgetedWarning","value":false},{"key":"toggleSplits","value":false},{"key":"toggleTransactionFilters","value":false},{"key":"warnOnQuickBudget","value":true}]

System Settings
ToolKit Version: 0.7.2
Operating System (Mac/Windows/Etc): Windows 8.1 and Windows 10
Browser (Chrome/Firefox/Safari): Chrome
Browser Version: 53.0.2785.89 m
Other Extensions:

This looks normal! ;)
Would you mind opening the chrome dev tools Settings > More Tools > Developer Tools and taking a screen shot of the error? Thanks!

Here you go!

Also, accidentally skipped my OS version in my original post.  This is happening on both Windows 8.1 and Windows 10.

Same here.

Same here toolkit version 0.7.2 , Chrome 52.0.2743.116 m (64-bit) , Windows 10 64bit

@R0MANI96 @casner @maisi

@R0MANI96 @casner @maisi
Is this all new to today (the reports worked yesterday)?
Also, do you guys have any incomes that do not have a payee?

@joshmadewell Reports worked like an hour ago for me.
I've added payees to all the incomes, but not all sub(split) incomes, if that matters.

@joshmadewell
I did, but I just went through them all and gave them all a payee.  I refreshed but it's still throwing an error.  The new reports worked yesterday with no problem.

Okay, I just checked and there was no code released to this report in 0.7.2 so I'm not sure what's causing this but I've written up a check to make sure the payeeData exists so the error should at least stop happening.
we might be able to hotfix this but there's no guarantee :(
@maisi subtransaction incomes shouldn't matter as long as the parent has a payee :D

I just got the error myself.
It appears to have something to do with the month. I went from ending my reports with August 2016 to ending them with September 2016. I had no issue until I did this, now I can't access the reports.

@nconantj we have a fix for this which will be in our next release :D

Is this any better for everyone in v0.7.3? It should be.

Yes! I can now open the Income and Expense. I still have a few cents of "unknown" income to track down, but at least the report shows.

All is good for me!

Great, I'll close this off then, thanks!


SVG

Add canvas export to SVG and Path2D layer

added test version

new test version

new test version width Gradient and Pattern support

release version


Create index.md & README.md tests

Where the root contains neither index.md nor README.md
Where the root contains both index.md README.md
Where the root contains only index.md
Where the root contains only README.md

Fixed long ago


Change Vend Sound to TG

About the PR

makes the vend sound like tg's (tg pr says its under cc0 1.0 and doesnt require attribution)
Why / Balance

tg one is way more satisfying
Media

https://github.com/space-wizards/space-station-14/assets/79580518/f9fd8acf-a26d-407e-8cb9-e6ec70e7d2fe

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl: Potato1234_x

tweak: Changed the vending machine sound to tg's one

the sounds cut off early i dont like it

the current ones cut off too late

the sounds cut off early i dont like it

I agree, it sounds incomplete.

the sounds cut off early i dont like it

it sounds like that because the sound is too short. Simply stretch the time a bit longer, doesnt even have to be much and it should immediately be a lot better

made it slower

I don't like how harsh the noise is. It's awfully sharp and hurts my ears.
Can you try muffling it a little?

I cant just construct the rest of the sound out of thin air...

i guess we close this :despair:


plugins {} block supports plugin requests for plugins already present on the classpath

This allows the following use cases using the plugins {} block:

request buildSrc plugin
// buildSrc/src/main/kotlin/build/MyPlugin.kt
package build

import org.gradle.api.*

open class MyPlugin : Plugin<Project> {

    override fun apply(project: Project) {
        // ...
    }
}

// buildSrc/build.gradle.kts
plugins {
    `kotlin-dsl`
    `java-gradle-plugin`
}

gradlePlugin {
    (plugins) {
        "my-plugin" {
            id = "my-plugin"
            implementationClass = "build.MyPlugin"
        }
    }
}

// build.gradle.kts
plugins {
    `my-plugin`
}

request plugin applied to parent project
// root/build.gradle.kts
plugins {
  id("foo") version "1.0"
}

// root/sub/build.gradle.kts
plugins {
  id("foo")
}

request plugin requested on but not applied to parent project
// root/build.gradle.kts
plugins {
  id("foo") version "1.0" apply false
}

// root/sub/build.gradle.kts
plugins {
  id("foo")
}

Requesting a version of a plugin already on the classpath is forbidden.
See https://github.com/gradle/kotlin-dsl/issues/426

Consider always failing when a version is specified in the child. It is redundant and any change to the parent would make the child fail anyway.

Very good point @oehme.
I made the changes.
In exchange for the classloader scope meta data we need to preserve the original plugin request version when a plugin resolution rule changes it. Because in the presence of a plugin resolution rule that call useVersion(..) the plugin requests will have a version set on the whole project hierarchy. And we don’t want to fail in this case.

Good idea, we'll want that for reporting purposes anyway I think. Maybe store the whole original request instead of just the original version?

Done

Comments addressed.
@oehme, @bamboo, could you have another look?

👍


updated recipes view

Nice work! Can't wait to see it with all the data from the backend!
We definitely need to make sure that we're writing more tests though.


add disable annotation detection for default managed cluster set

add disable annotation detection for default managed cluster set, ensuring users have property to modify default managed cluster set spec
Signed-off-by: ycyaoxdu<EMAIL_ADDRESS>

@qiujian16  @ldpliu  please help to review.

/retest

/approve
/lgtm