Update Service Aggregation Methods to use SQL

Purpose
Displaying the number "Totals for Service VMs" on the /service/explorer/:id page is expensive at scale.  The following table rows:

CPU
Memory
Disk Count
Disk Space Allocated
Disk Space Used
Memory on Disk

Currently requiring load in each hardware record for each vm record, and sometimes each disk record for those hardware records, causing a huge N*N(N)? + 1™ when a service has a lot of VMs.

This is going to be a long, almost blog-post like, explanation here, so hold on to your butts...
How we currently aggregate these stats...
Taking a step back and looking a particular example from one of the table rows above, to calculate (aggregate) the number of CPUs for a given service currently, you would call my_service.aggregate_all_vm_cpus and get returned the number of CPUs that are used by ALL VMs that were created using this service.  What happens under the hood is the following:

my_service.all_vms

This is a virtual_has_many, but basically is makes use of ancestry on services to return all of the VMs that are associated with this service , and any other service that is a descendant of this one (uses the .direct_vms method on each service and child service in app/models/service.rb to do this).

.map(&:hardware)

In reality, we are using a virtual_column here, but because it isn't making use of any arel, we are effectively doing a map.

.inject(0) {|sum, hw| sum += hw.cpu_total_cores.to_i

With anything having to do with Disks, we are also effectively adding another .flat_map(&:disks) following the .map(&:hardware) step, which then totals up the values from the disk records.  As you probably can probably guess, this is extremely expensive with a service with many VMs.
Representing the same thing with SQL
To represent the aggregate_all_vm_cpus as a column in SQL, you can represent Service.select(:id, :aggregate_all_vm_cpus).first as the following:
SELECT id,
      (
       --- Return only the SUM of all of the CPU cores to the main query (`.inject(0) { ... }`)
       SELECT SUM(hardwares.cpu_total_cores)

       --- Make sure we alias the table name so it doesn't conflict with the outer one
       FROM services aggregate_all_vm_cpus_services

       --- Join on services_resources that have a type of 'VmOrTemplate' (part of `.direct_vms`)
       JOIN service_resources ON service_resources.service_id = aggregate_all_vm_cpus_services.id
                             AND service_resources.resource_type = 'VmOrTemplate'

       --- Include vms and their associated hardware (`.map(&:hardware)`)
       JOIN vms               ON vms.id = service_resources.resource_id
       JOIN hardwares         ON hardwares.vm_or_template_id = vms.id

       --- Include services that are the current service, and descendants of that service (`.all_vms`)
       WHERE (("aggregate_all_vm_cpus_services"."id" = services.id
               OR "aggregate_all_vm_cpus_services"."ancestry" ILIKE CONCAT(services.id, '/%'))
              OR "aggregate_all_vm_cpus_services"."ancestry" = CAST(services.id AS VARCHAR))) AS aggregate_all_vm_cpus
FROM services
WHERE id = <my_service.id>;

This is not ideal, and should not be used when against a index action, as this nested SELECT would be run individually for each service, but for a show, it is about a 20ms nested select in the query for a service with 1800 VMs associated with it and it's children.
To do this for Disk Count, another join would need to be done for disks against the hardware records.
Moving this into virtual_attributes
Part the reasoning behind doing this as a nested SELECT statement is it allows it to be used as a virtual_attribute/virtual_column.  This allows it to be queried against and selected as if it were a column on the services table:
irb> Service.select(:id, :aggregate_all_vm_cpus).where(:id => 123).first.attributes["aggregate_all_vm_cpus"]
=> 12345

And multiple of these can also then be used together in a single query:
irb> Service.select(:id, :aggregate_all_vm_cpus, :aggregate_all_vm_disk_count).where(:id => 123).first.attributes
  Service Load (146.1ms)  SELECT  "services"."id", ((SELECT ...
=> {"id" => 123, "aggregate_all_vm_cpus" => 12345, "aggregate_all_vm_disk_count" => 4321}

Which now takes a fraction of a second instead of multiple seconds when grabbing each record individually.  This did require modifying the underlying methods in order to check for the existence of the attribute before aggregating in ruby.
Working it into /service/explorer/:id
For any of these new virtual_attribute arel methods to actually have an affect, they need to be included in the select in some way, otherwise they will just default to the original N+1 method of aggregating the data.  This is done when calling get_node_info in the if params[:id] of the explorer action, which then calls show_record, then identify_service, then identify_record (defined in ApplicationController::CiProcessing), and finally calling find_by_id_filtered (defined in the main ApplicationController.
For simplicity, the with_aggregates scope was created to make including the columns easier.  This needed to be added to the query that returned the record that was acted against in the views so the aggregates aren't recalculated, so modifying find_by_id_filtered was necessary to prevent this.
Metrics
The following metrics should be a clear indication of the improvements provided by this change.
Before

ms
queries
query (ms)
rows

32,534.7
278
2,922.5
74,540

After

ms
queries
query (ms)
rows

15,958.3
44
1,304.6
36,626

Considerations and Mentionables

Truth be told, we are technically moving the N+1 here from the application into the database, it just so happens that the DB is much more efficient at running this kind of stuff than the application is, and doesn't have nearly the object overhead that comes with running this in the DB.  That said, there will be probably a bit more stress put on the DB to execute these queries, but nothing in the sense of "multi second table locking", and overall it should be less of an impact on resources.
These are DEFINITELY not queries that you want to run in a index type fashion, as they will not scale. This is because the nested selects will individually run for each service record returned.  In this case, it will end up in a tying up a process, so these methods need to be used with care.
Optimally, sharing the JOINs here would be ideal as they would prevent making multiple loops in the database for each aggregate column.  I was originally under the impression that the current UI doesn't facilitate for something like that easily, but it might be possible based on how I have implemented the scopes for this and the changes to the controller.  This would be a minor improvement at this point, and the virtual_column arel improvements might still want to be included regardless.
The overwriting/modifications of find_by_id_filtered is ugly at best.  Unfortunately, we are inheriting a lot from the ApplicationController when finding records for "show" routes.  I was mostly going for allowing for additions (scopes on the finder), and overriding as few methods as possible when doing so.  find_by_id_filtered should still be the method that is used though at the end of the day and make sure we are finding only what the user has access to, and splitting it up would cause more chances for this filtering to be done incorrectly.

Something I noticed while working on that method is it makes two calls to the same record.  One for the find_by, and the other when tossed into Rbac.filtered.  Furthermore, this also blows away any scopes that might have been applied in the first finder find_by when tossed into Rbac, meaning I had to apply the :named_scope to Rbac or it wouldn't take.  Since normally this is used with just a model's class and an ID passed in, this is fine, but I found it confusing myself when trying to apply the scope originally to the first query.

@miq-bot add_label performance, ui, refactoring

I had added #11300 to help delegating to methods like these

Updated the code to fix cops.  Nothing has changed with the metrics, just breaking up the main method so rubocop is satisfied.

@miq-bot remove_label wip

@Fryguy please review when you have a chance.

I have a lot of reading to catch up on this one.  On the VERY casual review I did without actually reading anything, I will say the following...please take it with a grain of salt and I'll give a much deeper reading on Monday (hopefully 😛 )
First, I don't like all the Arel...I find if we need to drop down that low, I'm wondering if we are doing something wrong.  Perhaps I'm wrong.  I find when we drop down to Arel the code gets extremely complex to read, and generally there is a simpler way to bring the data back to Ruby efficiently and do the calculation there.  Yeah, it's a little slower or uses a little memory, but the maintenance value is MUCH easier.  For example, trying to get things into subselects with groupings and aliases, when a simple .includes(:hardwares).each { |x| calculate } will do.  Not sure...again I didn't read everything here (it's going to take a while 😄 )
Now, as far as aggregates go, it took me a while to figure out that you aren't talking about AggregationMixin.  In there we do optimized aggregation, but it doesn't look nearly as complicated as it does here.  Perhaps that's because of the ancestry layer, but I'm not sure yet.  I don't know why Service::Aggregation was built the way it was when we have AggregationMixin.  Have you reviewed AggregationMixin to see if it could be reused or if the techniques in there could be reused?

@Fryguy this looks good to me - anything on your side.
Maybe @chrisarcand has an opinion?

Yeah, it's a little slower or uses a little memory, ...

Working this in a more formal "report", but for now, the before and after for memory is about 300MB -> ~6MB with this change.

@Fryguy Responding to the rest of your comments:

I don't know why Service::Aggregation was built the way it was when we have AggregationMixin. Have you reviewed AggregationMixin to see if it could be reused or if the techniques in there could be reused?

I have no idea why we didn't use that in for Service, but I did notice that it existed after Keenan pointed it out.  I don't know if using that would be useful, as it seems like some of the columns in there are different from the ones aggregated in Service::Aggregation, and I am not sure if that would mess with the other models that include that module.  I can do a spike in switching to that to see if it would be of some use.

For example, trying to get things into subselects with groupings and aliases, when a simple .includes(:hardwares).each { |x| calculate } will do.

Not sure it is that simple.  To start, we are doing aggregations from the service level, meaning we are accessing VMs through the all_vms call.  Since this is using MiqPreloader's preload_and_map on the subtree, and direct_vms calls a collect on :resource, so this probably trashes any rails relationships that we have and allowing us to use includes, which is why I built it into the the arel that was generated.  From @kbrock 's comments, it should be noted that we can't use the scopes from ancestry either since they reference self.id in them.
Avoiding the includes also avoids instantiating all of these heavy objects just to calculate these numbers.  I did some light testing in with basically the following two statements:
irb> Service.where(:id => 12345678).first.aggregate_all_vm_cpus
irb> Service.where(:id => 12345678).with_aggregates.first.aggregate_all_vm_cpus

Here are the results when running it through MemoryProfiler:
Before
irb(main):001:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).first.aggregate_all_vm_cpus }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 276579199 bytes (2580575 objects)
Total retained:  25413864 bytes (85328 objects)
=> nil
irb(main):002:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).first.aggregate_all_vm_cpus }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 111997928 bytes (1264737 objects)
Total retained:  272 bytes (3 objects)
=> nil
irb(main):003:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).first.aggregate_all_vm_cpus }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 111997928 bytes (1264737 objects)
Total retained:  272 bytes (3 objects)
=> nil
irb(main):004:0> ActiveSupport::NumberHelper.number_to_human_size 276579199
=> "264 MB"
irb(main):005:0> ActiveSupport::NumberHelper.number_to_human_size 111997928
=> "107 MB"

After
Total allocated: 7160726 bytes (65234 objects)
Total retained:  1575598 bytes (8304 objects)
=> nil
irb(main):002:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).with_aggregates.first.aggregate_all_vm_cpus }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 104832 bytes (1554 objects)
Total retained:  0 bytes (0 objects)
=> nil
irb(main):003:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).with_aggregates.first.aggregate_all_vm_cpus }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 104494 bytes (1552 objects)
Total retained:  0 bytes (0 objects)
=> nil
irb(main):004:0> ActiveSupport::NumberHelper.number_to_human_size 7160726
=> "6.83 MB"
irb(main):005:0> ActiveSupport::NumberHelper.number_to_human_size 104832
=> "102 KB"

The changes in memory usage after the first on each of those can probably be attributed to the ActiveRecord query cache being enabled.  If we change out aggregate_all_vm_cpus for aggregate_all_vm_disk_count, which now requires loading the disk records for each hardware record, then we have the following:
Before
irb(main):001:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).with_aggregates.first.aggregate_all_vm_disk_count }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 7160686 bytes (65233 objects)
Total retained:  1575598 bytes (8304 objects)
=> nil
irb(main):002:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).with_aggregates.first.aggregate_all_vm_disk_count }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 104792 bytes (1553 objects)
Total retained:  0 bytes (0 objects)
=> nil
irb(main):003:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).with_aggregates.first.aggregate_all_vm_disk_count }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 104454 bytes (1551 objects)
Total retained:  0 bytes (0 objects)
=> nil
irb(main):004:0> ActiveSupport::NumberHelper.number_to_human_size 7160686
=> "6.83 MB"
irb(main):005:0> ActiveSupport::NumberHelper.number_to_human_size 104454
=> "102 KB"

After
irb(main):001:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).first.aggregate_all_vm_disk_count }.pretty_print(report); report.rewind; put
s report.readlines[0..1]
Total allocated: 375591683 bytes (3884256 objects)
Total retained:  25417360 bytes (85373 objects)
=> nil
irb(main):002:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).first.aggregate_all_vm_disk_count }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 211416769 bytes (2570130 objects)
Total retained:  272 bytes (3 objects)
=> nil
irb(main):003:0> report = StringIO.new; MemoryProfiler.report { Service.where(:id => 12345678).first.aggregate_all_vm_disk_count }.pretty_print(report); report.rewind; puts report.readlines[0..1]
Total allocated: 211416769 bytes (2570130 objects)
Total retained:  272 bytes (3 objects)
=> nil
irb(main):004:0> ActiveSupport::NumberHelper.number_to_human_size 375591683
=> "358 MB"
irb(main):005:0> ActiveSupport::NumberHelper.number_to_human_size 211416769
=> "202 MB"

I will let you be the judge if that is "a little memory", but this is only a portion of the request that I am profiling here.

Rebased with the following changes:

Added documentation for new methods (in a separate commit for now)
Removed unused option
Removed unused join on partitions
Removed need to pass around aggregation_sql variable to other methods

kicked the build. think it was unrelated.

I will let you be the judge if that is "a little memory", but this is only a portion of the request that I am profiling here

Sorry, I when I mentioned "a little memory" I was talking about the delta over alternate approaches.  By that I mean, given your change gets it from 300MB to 6MB, if there is an alternate approach that is easier to maintain, but only gets it from, say, 300MB to 10MB, then it might be better to use that one even though it's 4MB more (that's the "a little memory" I meant)

@Fryguy So that is where I am not sure how you can do this in Ruby without resorting to some kind of SQL solution.
What exists now is an N+1 for fetching all of the objects, which is slow (strictly because of the individual SQL calls), and uses a lot of memory in the ruby process because we are instantiating the those objects with all of there column data to perform the data aggregations.  If we were to use includes, we remove the N+1, sure, but we still instantiate the same number of objects to build those includes and should, in theory, use the same amount of memory.
We can limit this possibly by reducing the number of columns we return with the includes, meaning there is less for the ruby process to interpret from the database (buildings strings on the database adapter), and less overall in the ActiveRecord objects themselves, but the default ActiveRecord weight of those objects probably won't get us below the 50MB mark when dealing with 1k+ vms per service.
The other problem with a includes solution in this case is Service is a polymorphic relationship with VMs, so it is tough to do includes from that level.  I am getting around it by hardcoding it in the aggregation arel, but I am unsure of a clean way to do it in ruby.

I have also looked into the AggregationMixin a bit and it doesn't seem like it is a clean swap in and replace, and would also require some view changes to make this work in the front end so we are calling the proper methods names that differ between AggregationMixin and ServiceAggregation.

@NickLaMuro @kbrock @Fryguy this  PR is getting a bit stale, I believe the open question / issue is if the time saving (32 seconds down to 16 seconds) and memory savings (300Mb to 6Mb) for the scale env tested are worth the complexity or not. I've not heard a simpler solution discussed that provides a comparable level of win, so it's either we go with the added complexity or move on and live with the current solution for now - unless I missed a viable alternative. Looking for closure on a non-trivial request. Thoughts?

No, AggregationMixin does not work. It just encourages N+1 / same thing we are doing. It is also going against other columns and not meant to go across resources.
The main problems (that this tackles)

sql to join to all children of a service. Ancestry assumes that every parent id will be loaded into ruby.  This is not really a simple :through, so I'm not sure if my dream of using virtual_aggregates would even work here.

Nick: if you think we could tear this out if we got a better way of doing joins or sums, then I'm all for going this route.

As far as being able to replace this in the future, I didn't really add much here that is needed in other places, so I don't why not, and this definitely something that I think should be done in the future since I think the overall pattern we are doing here is a bit flawed (kinda what I was getting at in "point 3" in the "Considerations" section in the description).  I see this as a better to be fetched as a "Query Object", but that was a bigger hammer that I didn't want to invest in when I put this together in the past.

@dmetzger57 I agree.  Although I don't really like all of the Arel, this PR does solve the problem, and we've not come up with anything better, so on that note, I am :+1: on this PR.
I believe @NickLaMuro said he had to fix something in this PR due to another PR getting merge which creates a logical conflict (as opposed to a hard git conflict).
Additionally, I found a minor typo 😝  base_service_aggrigation_join should base_service_aggregation_join

@Fryguy fixed, though, I might do some quick performance tests with these changes just to be sure.

New Metrics:
Before

ms
queries
query (ms)
rows

37,057
7,673
3,018
74,323

40,147
7,486
2,724
102,413

40,239
7,486
2,701
102,413

41,698
7,486
2,701
102,413

41,160
7,486
2,688
102,413

After

ms
queries
query (ms)
rows

18,691
277
2160
36,409

20,475
91
1227
64,499

21,338
91
1083
64,499

21,230
91
1096
64,499

21,482
91
1090
64,499

The failing test is due to a failure on master which is now resolved.  Merging.

Helps performance of issues reported in https://bugzilla.redhat.com/show_bug.cgi?id=1403509

Darga Backport conflict:
$ git cherry-pick -e -x -m 1 bbe42e0
error: could not apply bbe42e0... Merge pull request #11502 from NickLaMuro/sqlize_service_aggregation_virtual_cols
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'

$ git status
On branch darga
Your branch is up-to-date with 'upstream/darga'.
You are currently cherry-picking commit bbe42e0.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

	modified:   app/controllers/service_controller.rb

Unmerged paths:
  (use "git add <file>..." to mark resolution)

	both modified:   app/controllers/application_controller.rb
	both modified:   app/models/service/aggregation.rb

$ git diff

diff --cc app/controllers/application_controller.rb
index bd67b54,e240c0d..0000000
--- a/app/controllers/application_controller.rb
+++ b/app/controllers/application_controller.rb
@@@ -2351,15 -2204,16 +2351,19 @@@ class ApplicationController < ActionCon
    end

    # Following 3 methods moved here to ensure they are loaded at the right time and will be available to all controllers
-   def find_by_id_filtered(db, id)
+   def find_by_id_filtered(db, id, options = {})
      raise _("Invalid input") unless is_integer?(id)

 -    db_obj = db.find_by(:id => from_cid(id))
 -    if db_obj.nil?
 +    unless db.where(:id => from_cid(id)).exists?
        msg = _("Selected %{model_name} no longer exists") % {:model_name => ui_lookup(:model => db.to_s)}
        raise msg
      end

++<<<<<<< HEAD
 +    Rbac.filtered(db, :conditions => ["#{db.table_name}.id = ?", id], :user => current_user).first ||
++=======
+     Rbac.filtered_object(db_obj, :user => current_user, :named_scope => options[:named_scope]) ||
++>>>>>>> bbe42e0... Merge pull request #11502 from NickLaMuro/sqlize_service_aggregation_virtual_cols
        raise(_("User '%{user_id}' is not authorized to access '%{model}' record id '%{record_id}'") %
                {:user_id   => current_userid,
                 :record_id => id,
diff --cc app/models/service/aggregation.rb
index aa45b35,f9860c4..0000000
--- a/app/models/service/aggregation.rb
+++ b/app/models/service/aggregation.rb
@@@ -42,26 -273,51 +273,75 @@@ module Service::Aggregatio
    end

    def aggregate_all_vm_cpus
++<<<<<<< HEAD
 +    all_vms.inject(0) { |aggregate, vm| aggregate += vm.cpu_total_cores.to_i }
 +  end
 +
 +  def aggregate_all_vm_memory
 +    all_vms.inject(0) { |aggregate, vm| aggregate += vm.ram_size.to_i }
 +  end
 +
 +  def aggregate_all_vm_disk_count
 +    all_vms.inject(0) { |aggregate, vm| aggregate += vm.num_disks.to_i }
 +  end
 +
 +  def aggregate_all_vm_disk_space_allocated
 +    all_vms.inject(0) { |aggregate, vm| aggregate += vm.allocated_disk_storage.to_i }
 +  end
 +
 +  def aggregate_all_vm_disk_space_used
 +    all_vms.inject(0) { |aggregate, vm| aggregate += vm.used_disk_storage.to_i }
 +  end
 +
 +  def aggregate_all_vm_memory_on_disk
 +    all_vms.inject(0) { |aggregate, vm| aggregate += vm.ram_size_in_bytes_by_state.to_i }
++=======
+     if has_attribute?("aggregate_all_vm_cpus")
+       self["aggregate_all_vm_cpus"]
+     else
+       all_vms.inject(0) { |aggregate, vm| aggregate + vm.cpu_total_cores.to_i }
+     end
+   end
+
+   def aggregate_all_vm_memory
+     if has_attribute?("aggregate_all_vm_memory")
+       self["aggregate_all_vm_memory"]
+     else
+       all_vms.inject(0) { |aggregate, vm| aggregate + vm.ram_size.to_i }
+     end
+   end
+
+   def aggregate_all_vm_disk_count
+     if has_attribute?("aggregate_all_vm_disk_count")
+       self["aggregate_all_vm_disk_count"]
+     else
+       all_vms.inject(0) { |aggregate, vm| aggregate + vm.num_disks.to_i }
+     end
+   end
+
+   def aggregate_all_vm_disk_space_allocated
+     if has_attribute?("aggregate_all_vm_disk_space_allocated")
+       self["aggregate_all_vm_disk_space_allocated"]
+     else
+       all_vms.inject(0) { |aggregate, vm| aggregate + vm.allocated_disk_storage.to_i }
+     end
+   end
+
+   def aggregate_all_vm_disk_space_used
+     if has_attribute?("aggregate_all_vm_disk_space_used")
+       self["aggregate_all_vm_disk_space_used"]
+     else
+       all_vms.inject(0) { |aggregate, vm| aggregate + vm.used_disk_storage.to_i }
+     end
+   end
+
+   def aggregate_all_vm_memory_on_disk
+     if has_attribute?("aggregate_all_vm_memory_on_disk")
+       # Avoids (poorly) "ERROR:  integer out of range" in postgres
+       self["aggregate_all_vm_memory_on_disk"] * 1.megabyte
+     else
+       all_vms.inject(0) { |aggregate, vm| aggregate + vm.ram_size_in_bytes_by_state.to_i }
+     end
++>>>>>>> bbe42e0... Merge pull request #11502 from NickLaMuro/sqlize_service_aggregation_virtual_cols
    end
  end

@chessbyte I don't think this should be backported to darga as it is taken care of in https://github.com/ManageIQ/manageiq/pull/13117

Backported to Darga via #13117

Euwe backport details:
$ git log -1
commit ca34e404e4d911084da5e20e7c8d61fc49cf75cd
Author: Jason Frey<EMAIL_ADDRESS>Date:   Tue Dec 13 20:45:50 2016 -0500

    Merge pull request #11502 from NickLaMuro/sqlize_service_aggregation_virtual_cols

    Update Service Aggregation Methods to use SQL
    (cherry picked from commit bbe42e02017aa9a9a16c1e52a318daafb3a7a7b7)

    https://bugzilla.redhat.com/show_bug.cgi?id=1411883
    https://bugzilla.redhat.com/show_bug.cgi?id=1411887


How do I remove double slash in URLs?

With this configuration:
    'url' => [
        'host' => null,
        'namespace' => '/api',
        'name' => 'api:',
    ],

URLs look like this, which is fine:
        "links": {
            "self": "https://api.loc8um.dev/api/locums/DA49890F-1CB5-46F8-A501-57E3B22C0BFA"
        }

However, I don't want/need the '/api' bit, so I change the configuration to this:
    'url' => [
        'host' => null,
        'namespace' => '',
        'name' => 'api:',
    ],

And the URLs look like this:
        "links": {
            "self": "https://api.loc8um.dev//locums/94A14168-4B12-403C-B403-E852C4E8B6BE"
        }

And it does the same thing if I set the namespace to '/', null or false too.
So, how do I stop it double slashing?

Confirmed this is a bug... looking at fixing it now.

Fixed in v0.11.5


Facebook Content Delivery Network

I had to whitelist this domain in order to download a pdf attachment from Facebook messenger.
cdn.fbsbx.com
Initial Googling of the domain screams "virus," but I'm not so sure that's the case.

This domain is related to Facebook, can I know in which adlist it's been included?

Since it's a Facebook domain, it will not be included in whitelist.txt instead it will be in optional-list.txt.
Thanks for reporting!


Allow token auth inside container

Follow-up to #2374 that fixes a minor regression that breaks the https://modal.com/playground app. This adds a test that was failing previously and succeeds with the change.
pytest test/client_test.py -k test_client_token_auth_in_container
# 1 passed, 18 deselected in 0.04s

git stash push modal/client.py
pytest test/client_test.py -k test_client_token_auth_in_container
# 1 failed, 18 deselected in 0.10s

Changelog

Fix a regression introduced in client version 0.64.209, which affects client authentication within a container.

🎉 🎉


The files required by running train.py are missing

Dear authors,
Nice study. I want to train the model on whole CATH 4.2 to reproduce the performance recorded in the paper. However, the files required by running train.py, such as train-CGraph10.pkl, are missing. Could you please upload the related files again?

The training data for CATH4.2 can be generated using a script from our previous work, available here: neurips19-graph-protein-design. Additionally, we have provided the append_cgraph function in data_preprocess.py to construct a contact-based graph for the preprocessed entries from CATH4.2.


Upgrade pytorch lightning to address vulnerability

Stack from ghstack (oldest at bottom):

-> #14
#10

Summary:

One of FLAVA example module's dependencies -- pytorch_lightning version 1.5.10 --- is being identified for vulnerability issue by GitHub (see https://github.com/advisories/GHSA-r5qj-cvf9-p85h).

This is fixed in v 1.6.0 -- https://github.com/pytorchlightning/pytorch-lightning/commit/8b7a12c52e52a06408e9231647839ddb4665e8ae

Test Plan:

Merge the PR and rerun open source checker internally.

@langong347 has imported this pull request.  If you are a Facebook employee, you can view this diff on Phabricator.


Enable set also :doc to vue-pdf , allow set the pdfDoc from outside f…

…or shared resource. (like v-for of vue-pdf)
Hi :)
I use vue-pdf with v-for  , and since it load it every time with new pdf document (pdf.js) - performance are bad with big number of pages.
I make this change and create separate "doc" prop.
The big change (except the part of loading the doc) is that when you set "doc" and not "src" when vue-pdf component will destroy it will not call to the pdf document destory.
its enable do things like:
<div class="viewer scrollbar-style">
    <pdf v-for="page in numPages" :key="page" :doc="pdfDoc" :page="page"  :style="{ width: zoom }" :id="'pdf'+ (page)" class="frame"></pdf>
</div>

interesting feature


month is not 1-indexed

a value of January puts a "0" in the input file, which is not valid (must be [1, 12])
@FlanFlanagan, do you know off the top of your head how to do this? It should be easy (i just don't know where) -- just print i + 1.

I will update the outdated

er output

This should be fixed with #101 . I'm going to close for now.


qiime2 to Phyloseq

Hi, I was running into an issue when I tried to convert QIIME2 to phyloseq. See below.
physeq<-qza_to_phyloseq(   features="mergerep.qza")
Error in validObject(.Object) : invalid class “otu_table” object:
Non-numeric matrix provided as OTU table.
Abundance is expected to be numeric.
Then I checked the Type and it shows "Sequence" instead of "Frequency".
SVs$type
[1] "FeatureData[Sequence]"
So I was wondering if there is any way to convert the type to Frequency before I import into phyloseq?
Thanks,

I think this means that your qza contains a fasta file of the representative sequences rather than feature abundances. If you want to import the sequences, you can do it manually with something like:
pdata<-
read_qza("mergerep.qza")$data %>%
refseq(.) %>%
phyloseq()

Off the top of my head, I'm not sure how helpful phyloseq would be with the representative sequences. I have the sneaking thought that this isn't what are actually wanting to do. I think you are wanting to import the table of feature abundances which means you need to go back to qiime and make sure you are getting the correct data to import.


French broken

French data file is corrupt:
Bugfix need:

place une not before nine
correct cinq

fixed with release 0.1.4 https://github.com/bastie/w2ni18n/tree/0.1.4

fixed with release 0.1.4 https://github.com/bastie/w2ni18n/tree/0.1.4


Provide content-length property

Hi, thanks for developing a Device Tree language server! I have been waiting for this. There is some information missing that CoC needs to function with the language server. I had a similar issue with the bitbake-language-server and it needs to be fixed from the language server side
[coc.nvim] Connection to server ginko_ls is erroring, Header must provide a Content-Length property.
[coc.nvim] Connection to server ginko_ls is erroring, Header must provide a Content-Length property.
{"{\"jsonrpc\"":"\"2.0\",\"method\":\"textDocument/publishDiagnostics\",\"params\":{\"diagnostics\":[{\"message\":\"Expected string\",\"range\":{\"end\":{\"character\":10,\"line\":1},\"start\":{\"character\":9,\"line\":1}},\"severity\":1}],\"uri\":\"file:///home/jan/Workspace/Yocto/build/workspace/sources/linux/arch/arm64/boot/dts/freescale/fsl-ls1028a.dts\"}}\u001b[2m2024-06-24T12:33:13.361769Z\u001b[0m \u001b[32m INFO\u001b[0m \u001b[2mtower_lsp::service::layers\u001b[0m\u001b[2m:\u001b[0m shutdown request receiv
ed, shutting down\nContent-Length: 38"}. Shutting down server.

Hi, thanks for your interest in the project!
I think this isn't related to not providing the Content-length property, but instead there seems to be something printed to stdout where it shouldn't be. In the log you provided, there is an ominous \u001b[2m2024-06-24T12:33:13.361769Z\u001b[0m \u001b[32m INFO\u001b[0m \u001b[2mtower_lsp::service::layers\u001b[0m\u001b[2m:\u001b[0m shutdown request received, shutting down\n appearing in the message. After that, the Content-Length property appears normally.
I'm wondering whether this is related to the usage of coc.nvim, as I cannot observe the error in VSCode (and I think people have used this successfully with Mason, but I cannot confirm that).
What seems odd to me is that there seems to be a shutdown request while receiving the textDocument/publishDiagnostics method. Do you know what the solution for bitbake-language-server was?

Afaik the bitbake-language-server did indeed not provide a content-length property. I don't know about the specifics though. If you are certain that it is provided, there could be an issue elsewhere. I will notify the developers of CoC about this. They are really active and helpful

I am not 100% certain, however from my perspective, this issue could be on either side. The only thing I'm observing is that there actually is a Content-Length property provided in the JSON output, however inter-mangled with some console output from the language server. This could be my problem, but this could also be an issue at coc.nvim, because the server is being shut down too early or similar.

coc.nvim uses vscode-jsonrpc to communicate with language server, the server should response with content-length header https://github.com/microsoft/vscode-languageserver-node/blob/33cbe5c87032cb53cf30c2b90f29d22a45ee6800/jsonrpc/src/common/messageReader.ts#L223-L227

I have finally found the time to try things out myself. I got the following output:
[coc.nvim] Connection to server dts is erroring, Header must provide a Content-Length property.
{"\u001b[2m2024-06-30t17":"26:58.978499Z\u001b[0m \u001b[33m WARN\u001b[0m \u001b[2mtower_lsp\u001b[0m\u001b[2m:\u001b[0m Got a workspace/didChangeConfiguration notification, but it is not implemented\nContent-Leng
[coc.nvim] Connection to server dts is erroring, Header must provide a Content-Length property.
{"{\"jsonrpc\"":"\"2.0\",\"method\":\"textDocument/publishDiagnostics\",\"params\":{\"diagnostics\":[{\"code\":\"expected\",\"message\":\"Expected one of /dts-v1/, /memreserve/, /include/, /delete-node/, /omit-if-n
o-ref/, '/', reference\",\"range\":{\"end\":{\"character\":1,\"line\":0},\"start\":{\"character\":0,\"line\":0}},\"severity\":1,\"source\":\"ginko_ls\"}],\"uri\":\"file:///Users/lukasscheller/.local/share/nvim/site
/pack/x.dts\"}}\u001b[2m2024-06-30T17:26:58.984067Z\u001b[0m \u001b[32m INFO\u001b[0m \u001b[2mtower_lsp::service::layers\u001b[0m\u001b[2m:\u001b[0m shutdown request received, shutting down\nContent-Length: 38"}.
Shutting down server.

Did you also see the first error message? If so then it seems that the client requests the workspace/didChangeConfiguration which I did not implement explicitly. Implementing this message results in the server being operational, so this would be a quick fix.

This is what I saw
[coc.nvim] Connection to server ginko_ls is erroring, Header must provide a Content-Length property.
[coc.nvim] Connection to server ginko_ls is erroring, Header must provide a Content-Length property.
{"{\"jsonrpc\"":"\"2.0\",\"method\":\"textDocument/publishDiagnostics\",\"params\":{\"diagnostics\":[{\"message\":\"Expected string\",\"range\":{\"end\":{\"character\":10,\"line\":1},\"start\":{\"character\":9,\"line\":1}},\"severity\":1}],\"uri\":\"file:///home/user/Workspace/OS/kernel/linux-somecompany/arch/arm64/boot/dts/freescale/fsl-ls1028ay.dts\"}}\u001b[2m2024-07-01T07:10:54.016267Z\u001b[0m \u001b[32m INFO\u001b[0m \u001b[2mtower_lsp::service::layers\u001b[0m\u001b[2m:\u001b[0m shutdown request received, shutting down\nC
ontent-Length: 38"}. Shutting down server.

I just saw the error message about posted above

So to conclude, I think that there is a way to quick fix the issue. However, I would like to find a more robust solution so I will investigate a bit further. If I don't find a good solution within the coming weeks, I will just roll out the quick fix.

I am not in a hurry. Do your thing! :)

Fixed in the latest master. Will be rolled out with the next update

Thanks, works well! :)
I created an AUR package btw, which builds directly from source
https://aur.archlinux.org/packages/ginko-git

Ah well, outline via :CocOutline or goto definitions doesn't work it seems

Thanks, works well! :)
I created an AUR package btw, which builds directly from source
https://aur.archlinux.org/packages/ginko-git

Thanks a lot! If you want to, you can open a PR with installation instructions in the readme

It seems to only have problems with .dtsi files. e.g.
https://github.com/torvalds/linux/blob/master/arch/arm64/boot/dts/actions/s700.dtsi
Adding install instructions for AUR packages is out of the scope of the README. Every Arch user knows how anyway. It is just a convenient way to compile the application from source.

It seems to only have problems with .dtsi files. e.g.
https://github.com/torvalds/linux/blob/master/arch/arm64/boot/dts/actions/s700.dtsi
Ah seems to be due to the C includes. You mentioned that in the README.
Adding install instructions for AUR packages is out of the scope of the README. Every Arch user knows how anyway. It is just a convenient way to compile the application from source.

Unfortunately, this is known and expected at the moment. These files contain C-style include directives (i.e., #include <some/path>) which ginko does not yet support. I know that this is a big pain point, because 99% of real-world device-trees contain these directives, so naturally a language server is only really useful if it also supports these directives. I'm working on it, but as I am maintainer of a different (and bigger) open-source project and have a full-time job as well, my time is limited. I am making progress, but this will be very slow (so don't expect anything any time soon).

It's fine :) I am grateful for anything to help when it comes to device trees, as I've always had problems understanding them. I know it know and can just delete the lines as long as I am editing the file.


add common label

change matchLabels
change namespace

Could we remove control-plane: controller-manager ununsed labels on all resources to avoid confusion?
Also, .spec.selector on Service will break since it still has older labels. But maybe Kustomize already does this by default, not sure.
Could have something like this, just so that it is explicit:
commonLabels:
- path: metadata/labels
  create: true

- path: spec/selector
  create: true
  version: v1
  kind: Service

- path: spec/selector/matchLabels
  create: true
  kind: Deployment

good idea, I think we should also apply it in lifecycle-manager @jakobmoellersap , what do you think?

``> commonLabels:

path: metadata/labels
create: true

path: spec/selector
create: true
version: v1
kind: Service

path: spec/selector/matchLabels
create: true
kind: Deployment

Nope this is not a feature request. The default configuration is: https://github.com/kubernetes-sigs/kustomize/blob/master/api/konfig/builtinpluginconsts/commonlabels.go
You can override this config by mentioning (only if default is not sufficient) in kustomization.yaml:
configurations:
 - ./config_override.yaml

commonLabels:

path: metadata/labels
create: true
path: spec/selector
create: true
version: v1
kind: Service
path: spec/selector/matchLabels
create: true
kind: Deployment

Nope this is not a feature request. The default configuration is: https://github.com/kubernetes-sigs/kustomize/blob/master/api/konfig/builtinpluginconsts/commonlabels.go
You can override this config by mentioning (only if default is not sufficient) in kustomization.yaml:
configurations:
 - ./config_override.yaml

You can think of adding ServiceMonitor in the overrides if you like.

Great thanks for this hint! @adityabhatia , I haven't notice this before, applied the code to your suggestions, please have a check.


stdenv: cross: updateAutotoolsGnuConfigScriptsHook unconditionally

Description of changes
We updateAutotoolsGnuConfigScriptsHook unconditionally in the native bootstrap; there's no reason to conditionalize it here.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Are you saying it already is an unconditional extra dep in the native bootstrap? Then I don't we need this bit at all!

Are you saying it already is an unconditional extra dep in the native bootstrap?

Not quite.  See 3b8e3c1f3de24.  It is a dependency of native stdenv, and therefore changes to it cause everything everywhere to rebuild.  However it is only a direct dependency (i.e. hooks applied) of those packages which are part of stdenv itself.
The point of my comment above was that this PR will not increase the number of rebuilds caused by future gnu-config version bumps.

https://github.com/NixOS/nixpkgs/commit/3b8e3c1f3de24472dd6dbd1fcdb3912a067bd70c#diff-bf918d7f48f6a34a4a1acb33373dee717cff2c3f5503a7ca41d90e5f289faf1eR615 doesn't that line make it part of the final stdenv / used by anything which is built with that stdenv?

3b8e3c1#diff-bf918d7f48f6a34a4a1acb33373dee717cff2c3f5503a7ca41d90e5f289faf1eR615 doesn't that line make it part of the final stdenv / used by anything which is built with that stdenv?

You're right.
I forgot that stdenv doesn't copy stdenv.mkDerivation doesn't prepend defaultNativeBuildInputs to the resulting nativeBuildInputs.  The additional dependencies are added in bash, not in Nix code.
I guess they are really propagated native build inputs, propagated from stdenv.

My only remaining concern is that setup.sh tries to be clever and filter out any nativeBuildInputs that can't be executed on the buildPlatform.
In spite of the fact that it's a shell script, Nix believes that gnu-config has a meaningful execution platform associated with it.
But I'm having a hard time making this happen.
I need to read findInputs() more closely.  Grr, I wish we weren't doing this stuff in bash scripts...

Ah, I see.
You can never change the buildPlatform using pkgs/stdenv/cross.
That explains it.


Implement Leaderboard Functionality

Is your feature request related to a problem? Please describe.
Add Leaderboard functionality, similar to what redis does. Needs to be highly space efficient.
Describe the solution you'd like
Should be functionally similar to Redis Leaderboard Overview

Assigning this to myself.

a beautiful sunset 🌥

a beautiful sunset 🌥

very beautiful

Hey @JyotinderSingh , is this closed or open ?

Hey @JyotinderSingh , is this closed or open ?

This is deprioritized for now.

This can be closed now @arpitbbhayani .


Bump version

Signed-off-by: Ben Dang<EMAIL_ADDRESS>

Do it to it! 👍

thanks @fishnix -- i'm gated by automation.  Need to get some time to debug the wercker setup.

Yeah I noticed that - I'm swamped too, if I get a chance I'll keep it on my radar

all good now!
:+1:


Create SATP server cluster with Kubernetes for redundancy and load balancing

To make SATP production ready we want to automatically add redundancy by setting up multiple versions of the gateway (for crash recovery too). Create a Kubernetes definition that keeps N SATP gateways for load balancing and redundancy.
Requirements:

Create Kubernetes cluster definition for N SATP gateway instances and N SATP user interfaces. Create N database instances
Expose endpoints and the Kubernetes Dashboard

hi

TBD later


feat: new config implementation, config option to allow leaking of secret values in logs

can you merge main in here


analyzing a lot of files at once (48) is bad.  Also analyzing a directory is bad too.

Analyzing 48 files...

Turns out you get this error if you supply a file name that doesn't exist as well.

ie. flutter analyze i_dont_exist.dart

Yeah the problem is that the error message we're getting from the dartanalyzer is in the main.dart file and by the time we try to read it we've already deleted it.

Is it having some sort of race with the creation/deletion of its temp directory?  Do you believe this is a problem with flutter analyze (in which case @Hixie) or the underlying analyzer (in which case @pq).

A quick fix might just be to move host.deleteSync(recursive: true) until after the processing of errorLines?

That would at least change the error message. But we should never be getting errors in the generated file regardless (which is why I put the delete first -- so that it would fail hard if we did).

So if I remove the deleteSync I actually get some valid errors.
The only one I think is different than others which don't break the system is this one:
The imported libraries '<redacted>_test.dart' and '<redacted>_test.dart' cannot have the same name '<redacted>_test' (/tmp/flutter-analyze-iKe27T/main.dart, line 44, col 1)

But we should never be getting errors in the generated file regardless

Should we add an existence check on flutter analyze targets?

@apwilson : is this failure repeatable?

I have a fix but in trying to verify it I ran into a problem whereby it turns out we're not checking several of our tests so I'm fixing that too.

Yes, happens every time.


Spring library seems to depend on Spring Data

Hello!
Here are my dependencies: (library version 2.4.0)
    compile "com.amazonaws:aws-xray-recorder-sdk-core"
    compile "com.amazonaws:aws-xray-recorder-sdk-aws-sdk"
    compile "com.amazonaws:aws-xray-recorder-sdk-spring"
    compile "com.amazonaws:aws-xray-recorder-sdk-apache-http"
    compile "com.amazonaws:aws-xray-recorder-sdk-slf4j"

Configuration is set up as per the AWS Documentation, and works for all other services I have implemented so far.
I am trying to implement X-Ray in a Spring application that doesn't connect to a database. This is the first service I've tried to add X-Ray to that has not worked.
I am getting the following error when I run my service:
Caused by: java.lang.IllegalArgumentException: warning no match for this type name: org.springframework.data.repository.Repository [Xlint:invalidAbsoluteTypeName]
	at org.aspectj.weaver.tools.PointcutParser.parsePointcutExpression(PointcutParser.java:319) ~[aspectjweaver-1.9.4.jar:na]
	at org.springframework.aop.aspectj.AspectJExpressionPointcut.buildPointcutExpression(AspectJExpressionPointcut.java:227) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.aspectj.AspectJExpressionPointcut.obtainPointcutExpression(AspectJExpressionPointcut.java:198) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.aspectj.AspectJExpressionPointcut.getClassFilter(AspectJExpressionPointcut.java:177) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.support.AopUtils.canApply(AopUtils.java:225) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.support.AopUtils.canApply(AopUtils.java:288) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.support.AopUtils.findAdvisorsThatCanApply(AopUtils.java:320) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.findAdvisorsThatCanApply(AbstractAdvisorAutoProxyCreator.java:126) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.findEligibleAdvisors(AbstractAdvisorAutoProxyCreator.java:95) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.getAdvicesAndAdvisorsForBean(AbstractAdvisorAutoProxyCreator.java:76) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator.wrapIfNecessary(AbstractAutoProxyCreator.java:347) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator.postProcessAfterInitialization(AbstractAutoProxyCreator.java:299) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsAfterInitialization(AbstractAutowireCapableBeanFactory.java:429) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	... 25 common frames omitted

After a bit of debugging, and making sure I didn't depend on any of the database related X-Ray libraries, I found that the library for Spring aws-xray-recorder-sdk-spring:2.4.0 depends on spring data classes:
Lines 69 to 71 of the com.amazonaws.xray.spring.aop.AbstractXRayInterceptor.java
    @Pointcut("execution(public !void org.springframework.data.repository.Repository+.*(..))")
    protected void springRepositories() {
    }

I feel that this should only be a dependency when using the database related X-Ray libraries. Is there any way around this?

Turns out there is an undocumented BaseAbstractXRayInterceptor.java class that is created for this exact purpose:
From com.amazonaws.xray.spring.aop.BaseAbstractXRayInterceptor
Why the normal AbstractXRayInterceptor is the default for the documentation confuses me. But either way, I hope this helps someone.

It appears that this class is not available in the 2.4.0 artifact? This class was added in January, and the last X Ray release in Maven was in November. Do you have a time line for release?

Hi @joodlehammond,
Unfortunately we cannot provide official timelines for our releases, but that change will be included in the next one! Feel free to "watch" this repo to get notified when we make it.

We release v2.5.0 Java SDK yesterday. So resolving.

Thanks guys, can't wait to try it out!


Tooltip on IE doesn't hide properly

IE Version: 11

Tooltip doesn't hide properly while moving around mouse pointer
while moving around mouse pointer, it adds vertical and horizontal scroll bars to the page

SAME issue

Did this ever get resolved?

Sorry guys, no this hasn't been resolved yet - I've stepped away from this project for a while now (so little time!).

Can you please fix this issue.

There is a fix for this in this post: https://github.com/benkeen/d3pie/issues/43


Not working

It's not working

Hi, I wonder what version of React Native did you try to use? Android compileSdkVersion escaped? What was the error you got? Could you give a little more information?


Add Japanese translations for chart node

Checklist

[x] I have read the contribution guidelines
[ ] Suitable unit/system level tests have been added and they pass
[ ] Documentation has been updated

[ ] Upgrade instructions
[ ] Configuration details
[ ] Concepts

[ ] Changes flowforge.yml?

[ ] Issue/PR raised on FlowFuse/helm to update ConfigMap Template
[ ] Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production

Labels

[ ] Backport needed? -> add the backport label
[ ] Includes a DB migration? -> add the area:migration label

Thank you for this merging it and releasing the new version! I tested the new version after installing it from npm. It works correctly.


drivers/usbdev/fs: fix issue between adb/fastboot/mtp and usbdev fs

Summary
Solve some problems in usbdev fs see commit message for details.
Impact
usbdev fastboot/adb
Testing
testing vela

Why is there this error?

arm-none-eabi-ld: /github/workspace/sources/nuttx/staging/libdrivers.a(usbdev_fs.o): in function `usbdev_fs_classsetup':
usbdev_fs.c:(.text.usbdev_fs_classsetup+0x50): undefined reference to `work_queue'
make[1]: *** [Makefile:211: nuttx] Error 1
make: *** [tools/Unix.mk:546: nuttx] Error 2
make: Target 'all' not remade because of errors.
/github/workspace/sources/nuttx/tools/testbuild.sh: line 370: /github/workspace/sources/nuttx/../nuttx/nuttx.manifest: No such file or directory
  Normalize stm32f103-minimum/adb

Please fix the build

arm-none-eabi-ld: /github/workspace/sources/nuttx/staging/libdrivers.a(usbdev_fs.o): in function `usbdev_fs_classsetup':
usbdev_fs.c:(.text.usbdev_fs_classsetup+0x50): undefined reference to `work_queue'
make[1]: *** [Makefile:211: nuttx] Error 1
make: *** [tools/Unix.mk:546: nuttx] Error 2
make: Target 'all' not remade because of errors.
/github/workspace/sources/nuttx/tools/testbuild.sh: line 370: /github/workspace/sources/nuttx/../nuttx/nuttx.manifest: No such file or directory
  Normalize stm32f103-minimum/adb

Please fix the build

Done, thanks~


Setup cant run windows 10

GPMDPError.zip

OS: Windows 10 anniversary update 14393.187
Issue Descriptions: Setup cannot install
Steps to Reproduce: run the setup. crash

Please disable your A/V and run as admin to work around this issue.
Basically your A/V is isolated the installer for whatever reason


Amd fix

UMD pattern doesn't work correctly with AMD modules because the values are exported before initialized correctly. These commits do the following things:

Move UMD pattern to the bottom of the function so that the exported values have all properties available
Return the same object for both Node.js require and AMD. Previously only the native Promise/Polyfill was returned for AMD.
Add usage instructions for AMD.

UMD pattern doesn't work correctly with AMD modules because the values are exported before initialized correctly.

My example in #8 works fine I checked it before post comment. AMD support was contributed by one of polyfill user (commit d59207b603af3ad4005a403cbda05fef056997dd) and I suppose it was checked too (PR #4). So I believe it should works.

It isn't necessary to move UMD pattern to bottom since NativePromise is initializing before pattern and Promise is function declaration that hoisting.
Polyfill was added to exports recently, just for test purposes. So no need export it for define. Exporting just Promise looks good to me. And for now it couldn't be changed for backward capability reason.
AMD usage example is good addition. Agree that's should be added to readme.

Hmm,
I'll send my test spec later on. I tested AMD support using Karma (Browsers: PhantomJS & Firefox).

Seems like I got misconfused with Node.js require returning an object and AMD returning a constructor function. The code works as it should. Apologies.


Support for non-editable columns

It would be cool if we can choose non-editable columns.
I've patched your widget like
element.find("td:not([data-editable='no'])").on('click keypress dblclick', showEditor)

and
element.find("td:not([data-editable='no'])").prop('tabindex', 1);

hi all,
i have tried to added this:
element.find('th[data-try!="false"]').on('click keypress dblclick', showEditor) (line 92)
element.find('th[data-try!="false"]').prop('tabindex', 1); (line113)
and set the column table as:
Tool Name
but i still couldnt edit any field from this column, is there anymore place i need to edit? kindly help on this item


Fix LayerNorm.

According to "attention is all you need", the formula
apply to sublayer is supposed to be LayerNorm(x + Sublayer(x)).
As mentioned in
issue#142,
this implementation results from the consideration of the problem in pytorch(see pytorch
issue#4320), which has
been fix in Fix standard deviation gradient.
related experience statistics
python train.py -data_pkl m30k_deen_shr.pkl -log m30k_deen_shr -embs_share_weight -proj_share_weight -label_smoothing -save_model trained -b 128 -warmup 12800 -epoch 40

Epoch 39
* original version
- (Training)   ppl:  355.27087, accuracy: 13.232 %, elapse: 0.691 min
- (Validation) ppl:  284.94428, accuracy: 11.988 %, elapse: 0.007 min

* follow formula in the paper
- (Training)   ppl:  297.18417, accuracy: 15.274 %, elapse: 0.696 min
- (Validation) ppl:  1478.94354, accuracy: 11.988 %, elapse: 0.007 min

co-contributor: @yichung279

Hi @zyex030640417 , thanks for the pr.
so what is the correct pytorch version you are using?
thanks.

The whole experience is based on pytorch==1.5.0.
BR

Hi @jadore801120 & @tony2037 , So earlier version (< 1.5) didn't experience this issue with NaN gradients? Does it only happen in the case of torch version > 1.5?  I am currently using attention mechanisms in same way and experiencing the NaN gradient issue.


Design issues in Category field

There are some design issues introduced in #264, which are really making the report issue page look messed up, and should be eradicated quickly.

Font of the selected category ("General" as in the above image) is not consistent with the rest of the page.
Selected category is not vertically centered for some reason, which makes the TextField look ugly.
"Category" button and selected category elements violate the Proximity principle of design. The separation between them should be no more than the separation of "App name/url" heading and the corresponding text field.

Fixed under https://github.com/OWASP/BLT-Flutter/pull/286


Nb/word counter

@0xrohitgarg Can you point to the audio asset used in this branch?

@codeAshu we have asset in separate repo:
https://github.com/video-db/videodb-cookbook-assets/raw/main/audios/twink.mp3


Botanist random colony

go to random planet to encounter Botanists that want to be transported to another colony for safety, after you rescued their team mate. the quest marker will vanish if you do not directly move to that location, and the botanists will stay on your ship forever

Details
reloading an earlier safe fixes it, but the bug reappears if you accept the quest again. it may very well be because I had the place their friend was held hostage, whipped clean before I accepted the quest to rescue the hostage.
Game Version
1.7.33
New Game
Yes
Reproduction Steps
I don't know

Some more details: the colony the botanists want to go is named "Waggoner Farm", in the "Cheyenne" system. I just googled for the quest and it spit out a  record on "game-maps.com", that helped my way to the location. Upon arriving, the botanists left the ship, thanked me and gave me my loan. The quest marker never showed up during the whole quest. And another oddity: when exiting my ship their voices turned into that weird "Google translate"-sound. They spoke normally inside the ship, and on the location where I picked them up, and the final location(Waggoner). The pickup happened on a icy moon in the "Vega" system. Can't rememember the name though, save game already removed from my circle.

This seem to be partially working now. The weird voices are gone, and the quest marker is added correctly to all targets, as well as to the correct quest giver on finish. But the marker is still not correctly set up on the quest giver on either starting dialogue for the quest, or auto init. This may very well be because of me not having started a new game since 8 or so weeks(NG+1, 400 hours). But since the quest has happened to me twice, and both time acted the same I guess they at least partially fixed it.


cannot download package from external repository

Verdaccio ver : 2.7.0
I have server 2012 R2 with verdaccio install on iisnode in internal network.
In our company we have a Nexus that open to npmjs.
In config.yalm i have config the uplinks to the nexus url:
uplinks:
nexus:
url: http://nexus.com/npm
packages:
access: $all
publish: $authenticated
proxy: nexus
My npm registy config to verdaccio.
I want that verdaccio auto download packages to store from nexus if the package don't exist
when i do npm install i saw in the verdaccio store the packages it try do download with .tmp but after seconds it deleted.
In iis logs i the see error:

http --> 200, req: 'GET http://nexus.com/npm/@angular%2Fflex-layout' (streaming)
http --> 200, req: 'GET http://nexus.com/npm/@angular%2Fflex-layout', bytes: 0/22906
http <-- 200, user: undefined(undefind), req 'GET /@angular/flex-layout, bytes: 0/4351
error--- unexpected error: getaddrinfo EAI_AGAIN registry.npmjs.org:443
Error: getaddrinfo EAI_AGAIN registry.npmjs.org:443
at Object.exports._errnoException (util.js:1018:11)
at errnoException (dns.js:33:15)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:76:26)
http <-- 500, user: undefined(undefined), req: 'GET<EMAIL_ADDRESS>error: internal error

Please help me to fix that.
Thanks!

The Full config file:
# # This is the default config file. It allows all users to do anything,
# so don't use it on production systems. #
# Look here for more config file examples:  # https://github.com/verdaccio/verdaccio/tree/master/conf
# # path to a directory with all packages
storage: ./storage auth:
htpasswd:
file: ./htpasswd
# Maximum amount of users allowed to register, defaults to "+inf".
# You can set this to -1 to disable registration.
#max_users: 1000
# a list of other known repositories we can talk to
uplinks:
nexus:
url: http://nexus.com/npm
packages:
'@/':
# scoped packages
access: $all
publish: $authenticated
proxy: nexus
'**':
# allow all users (including non-authenticated users) to read and
# publish all packages
#
# you can specify usernames/groupnames (depending on your auth plugin)
# and three keywords: "$all", "$anonymous", "$authenticated"
access: $all
# allow all known users to publish packages
# (anyone can register by default, remember?)
publish: $authenticated
# if package is not available locally, proxy requests to 'npmjs' registry
proxy: nexus
`# log settings
logs:

{type: stdout, format: pretty, level: http}
#- {type: file, path: verdaccio.log, level: info}
listen:

http://verdaccio.com:4873/         # default value`

Ok, I see more clearly your issue. @*/*' only will route any scoped packages through your nexus uplink. So, any other dependency, eg: react or babel-core will try to access the default one which is npmjs even if is not defined in your config file. So, if you want to solve your issue, either all resolve your traffic goes through '**': through nexus uplink or allow verdaccio to connect to registry.npmjs.org.
more info http://www.verdaccio.org/docs/en/packages.html

You can see that i have config the file right:
'**':
proxy: nexus
and it not working

Sorry, I did not see on my mobile the rest of the config file.
packages:
  '@*/*':
    # scoped packages
    access: $all
    publish: $authenticated
    proxy: nexus
  '**':
    # allow all users (including non-authenticated users) to read and
    # publish all packages
    #
    # you can specify usernames/groupnames (depending on your auth plugin)
    # and three keywords: "$all", "$anonymous", "$authenticated"
    access: $all
    # allow all known users to publish packages
    # (anyone can register by default, remember?)
    publish: $authenticated
    # if package is not available locally, proxy requests to 'npmjs' registry
    proxy: nexus

Please do the following https://github.com/verdaccio/verdaccio/issues/571#issuecomment-366382208 and display more debug information.


drop-down list

i hope fortune-sheet have drop-down list,same as it in excel
now,when i import xls, its drop-down list will dispear

能提供一些截图吗，不是很能理解需要什么样的下拉列表

像这样的

增加进TODO-LIST

drop-down list is supported in the latest version

Hi @lvjing-tuzhi Can you please help with the sample code or any documentation of the drop-down list.


Translation: Do not translate dynamic values

Problem
You can see in this commit that the dynamic value {beneficiaryAddress} was translated:
https://github.com/KlimaDAO/klimadao/commit/4588be697fb9f82f91dd5f00f15cbe6cd5307fa6#diff-97df1968ed9ec8222e8c69b2b96add299b947ff53176f565c9c00a121b3f7b3aR659-R663
Instead, dynamic values should remain and look like this:
https://github.com/KlimaDAO/klimadao/commit/4588be697fb9f82f91dd5f00f15cbe6cd5307fa6#diff-b93c89149e99aa09e66d429f3ea29d06599180e77984361ba575d54bd0505df3L665-R669
=> {beneficiaryAddress} should stay
Solution
Re-add the dynamic value {beneficiaryAddress}  on translation.io for each language again 🙂
Double check if more dynamic values have been changed.

Re-added missing dynamic values for each language and reverted their translation on translation.io.
I also alerted the translators to this so that they are aware next time. This issue is resolved.

Thanks. Closing


Improve post/reply edit history

Diffs for edit history

HTML based? Highlighting? List of fields changed?

Mark edits significant for 'unread' purposes (such that users have to jump back to reread it)
View all edits in a post (or thread, see #5)

The display was improved recently in #512 in anticipation of moderators. (Mostly unrelated to the issues mentioned here.)


Minimal changes to allow manual re-initialization of NodeJS process

This PR aims to solve issue #130 by creating an interface to terminate and initialize the bridge's underlying NodeJS process and threads responsible for the communication.
To do this, I moved the calls responsible for referencing the bridge classes instances to the initialization function, refactored EventLoop and EventExecutorThread's constructors to re-initialize its properties, changed the connection logic so it does not stop upon exception or process termination, and implemented a terminate() function in __init__.py.
With this change, the following behaviour is possible:
import javascript

javascript.init()
javascript.eval_js('console.log("Hello from 1st NodeJS process!")')
javascript.terminate()

javascript.init()
javascript.eval_js('console.log("Hello from 2nd NodeJS process!")')
javascript.terminate()

Upon running this code, each time we terminate() and init(), a new NodeJS process is spawned and the bridge switches to communicate with it. Note that the first eval_js call can be done without calling init() because this function is called within the module's initialization. The function itself checks if the bridge has been previously initialized, so calling it again to ensure it is up is fine.
Most importantly, these changes should not break the current usage of the bridge. I have verified this by running test.py and test_general.py, and no exception is raised.

I have only noticed one caveat about these changes regarding the global variables exposed by the module. When running the bridge interface after re-initializing the NodeJS process at least once, the following code does NOT work:
from javascript import globalThis
# ...
print(globalThis.Date().toLocaleString())

However, this DOES work:
import javascript
# ...
print(javascript.globalThis.Date().toLocaleString())

This is because the variable we get from from javascript import globalThis can only point to the global context of the first process as that's the reference that exists during the module import. On the other hand, calling javascript.globalThis can return the updated reference after the bridge re-initializes. I suspect this is also the case for console and RegExp as they are exposed with the same logic.
A cleaner way to solve this would be to write a getter for each of these variables and expose that instead of the references to variables themselves. For example, we can write in __init__.py:
def get_global_this():
    return config.global_jsi.globalThis

This however would change the interface to the bridge (i.e., we would have to call get_global_this().Date() in the code above), therefore I did not commit it in this PR. Please let me know if this is desirable though.

Thanks for working on this. As the lib was not designed around the Node.js process being stopped, all variables and all state depending on the previous node process will stop working and throw errors. Not just globalThis, but require(), and any other function currently stored on the Python side depending on a reference in JS land. This may or may not be handled by the user but if they intend to use this new API call, I think it's OK to assume the user will have written their code in a way to handle this (re-doing imports, using import javascript, etc.)
However I don't understand the removal of the error handling when the Node process is killed or terminated. This is intended to prevent later errors and other undefined behavior. If the Node process crashes this normally means the exception happened outside of a call done by Python as we handle most errors on object doing property accesses and calls. Can you explain this change?

However I don't understand the removal of the error handling when the Node process is killed or terminated. This is intended to prevent later errors and other undefined behavior. If the Node process crashes this normally means the exception happened outside of a call done by Python as we handle most errors on object doing property accesses and calls. Can you explain this change?

Thank you for the review! I have initially removed the error handling because that was inadvertently terminating the new Node process. However I took a second look at it and figured out a better way to handle this by checking the current Node process status and adding calls to sendQ, as it's done with other cases. With this I restored the stop() call in the error handling. Please let me know what you think about it!

Maybe this could be used in a mechanism to restart NodeJS in case it crashes or the JS app there ends up in an invalid state?
I don't think I need multiple instances, but am using the bridge in a long running Python server, to use a JS/TS lib that does websocket calls to a service. We haven't had problems, but was just thinking that maybe something like this could help if we get a need to restart the bridge / node some day.
I will be needing multiple worker processes though but I think those end up with separate Python interpreters and thus bridge instances & node processes anyhow.

Maybe this could be used in a mechanism to restart NodeJS in case it crashes or the JS app there ends up in an invalid state?

Yes, this is exactly the use case for this change. Note that this does not add support for parallel NodeJS process, but it allows the process to be re-initialized.
With this change, it is now possible to wrap every (independent) JS call with javascript.init() and javascript.terminate(), so the process is always re-initialized when needed. I assume it's also possible to catch bridge exceptions and handle them with terminate() followed by init(), to restore the process for subsequent JS calls.

@extremeheat I have just added two paragraphs in docs/python.md about controlling the node process and the API calls, as well as describing the caveat I mentioned in this PR. Please feel free to edit my text and to bring some of it to README.md, if you see necessary.

Thanks for the contribution! I agree with the comment about improving error handling but that can be handled outside the PR and may require some additional thought (as we'd have the same problem of existing state being unrecoverable, so something would need to be done to deal with this).


Play sound archive

Is there a way to play an audio file, for example mp3, when opening a scene?

Sure. You can use the API's on method to add a listener for scenechange events and use that to trigger custom code that plays an audio file.

Thank you! ... A client who owns a hotel asked me if I could add a voice that describes the rooms when the visitor enters each of them.


Add validation that we haven't broken apis.json

First commit adds a check in Travis (which currently goes red).
Second commit fixes apis.json so it goes green.

I'm going to merge this without review, so I can rerun autosynth and clear out all the issues.


Running the example code works briefly and then generates a stack trace and exits

Describe the error
Running the example code works briefly and then generates a stack trace and exits
Code and Error
$ manimgl example_scenes.py OpeningManimExample

Error:
sh: latex: command not found{c}\quad \\\quad \\\end{array}\right]"
[09:38:08] ERROR    LaTeX Error!  Not a worry, it        tex_file_writing.py:112
                    happens to the best of us.
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/manimgl", line 8, in <module>
    sys.exit(main())
  File "/Users/chuck/git/manim/manimlib/__main__.py", line 25, in main
    scene.run()
  File "/Users/chuck/git/manim/manimlib/scene/scene.py", line 131, in run
    self.construct()
  File "/Users/chuck/git/manim/example_scenes.py", line 29, in construct
    IntegerMatrix(matrix, include_background_rectangle=True),
  File "/Users/chuck/git/manim/manimlib/mobject/matrix.py", line 205, in __init__
    super().__init__(matrix, element_alignment_corner=element_alignment_corner, **kwargs)
  File "/Users/chuck/git/manim/manimlib/mobject/matrix.py", line 100, in __init__
    self.add_brackets(bracket_v_buff, bracket_h_buff)
  File "/Users/chuck/git/manim/manimlib/mobject/matrix.py", line 145, in add_brackets
    brackets = Tex("".join((
  File "/Users/chuck/git/manim/manimlib/mobject/svg/tex_mobject.py", line 208, in __init__
    super().__init__(full_string, **kwargs)
  File "/Users/chuck/git/manim/manimlib/mobject/svg/tex_mobject.py", line 58, in __init__
    super().__init__(
  File "/Users/chuck/git/manim/manimlib/mobject/svg/svg_mobject.py", line 72, in __init__
    self.init_svg_mobject()
  File "/Users/chuck/git/manim/manimlib/mobject/svg/svg_mobject.py", line 99, in init_svg_mobject
    self.generate_mobject()
  File "/Users/chuck/git/manim/manimlib/mobject/svg/svg_mobject.py", line 114, in generate_mobject
    file_path = self.get_file_path()
  File "/Users/chuck/git/manim/manimlib/mobject/svg/tex_mobject.py", line 87, in get_file_path
    file_path = tex_content_to_svg_file(
  File "/Users/chuck/git/manim/manimlib/utils/tex_file_writing.py", line 81, in tex_content_to_svg_file
    create_tex_svg(full_tex, svg_file, compiler)
  File "/Users/chuck/git/manim/manimlib/utils/tex_file_writing.py", line 115, in create_tex_svg
    with open(root + ".log", "r", encoding="utf-8") as log_file:
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/3j/kxrrqr310jzd37rtx2cdf1zm0000gn/T/Tex/c3372cf5b2620435.log'
[Chucks-MBP:manim (master)]527$ ls -l /var/folders/3j/kxrrqr310jzd37rtx2cdf1zm0000gn/T/Tex/
total 8
-rw-r--r--  1 chuck  staff  625 Dec 19 09:38 c3372cf5b2620435.tex

Environment
MacOS Ventura 13.0:
ManimGL v1.6.1: master
Python 3.10.7:

The instructions for MacOS simple say to use 'brew install ffmpeg mactex'. This does not seem to make LaTeX available anywhere, which may be the issue. Perhaps this is just a documentation problem?

I resolved this by augmenting my PATH. Apparently, brew doesn't put the TeX stuff in the usual brew place and doesn't seem to let you know. I believe the docs here should suggesting adding:
PATH=$PATH:/Library/TeX/texbin
to startup files.

On my macOs Sonoma, running source ~/.zshrc fixed this issue. Seems that sometimes, simply doing a brew or pip install doesn't update the PATH.


deleting modules

one step is to delete as "not show" and another one is "reset"

rely on JB's own removal func


Project Dead?? - Need a maintainer?

Assuming this project is dead since its not been maintained for quite some time (and there is no chrome extension in the store anymore?)
Are you looking for someone to maintain it?
I have some ideas  - was planning on forking it for myself - but I'd be happy to take this over?

@gareth-johnstone we can talk via gmail<EMAIL_ADDRESS>


Everything settings don't persist

After opening Everything for the first time, the Everything.db and Everything.ini hardlinks get broken, turning into normal files. I am not sure why this is happening, but I'm guessing Everything deletes and recreates them.
If it turns out to be Everything's fault, one option might be saving the settings to APPDATA, using the -app-data switch, or even using the installer, as it was done with Ditto

Closing as this issue is stale. Please reopen if this is still an issue.


Update Tableland Go client

This is in theory pretty straight forward, but uncovered this problem with tidying Go dependencies https://github.com/tablelandnetwork/go-tableland/issues/531. I'd like to see if we can get that resolved before merging this, so leaving this in draft for now.

Ok go that dependency issue resolved. This is ready now.


if<EMAIL_ADDRESS>@conn.disconnect => connection was forcibly closed by the remote host

How to check of connection was closed by remote host?
I'm trying to make a thread, that receive stomp messages and another stuff and reconnect, when the connection is down. But I could not determine, when it is down.
<EMAIL_ADDRESS>=> true
@conn.poll => nil (as if queue has no messages)

So, when I interrupt and try to disconnect, I got exception.
@conn.disconnect if<EMAIL_ADDRESS>=>
An existing connection was forcibly closed by the remote host..

I don't want to remove this exception. I want to know if connection was already closed before disconnect.

Now I use the hash:
hash = {
  :hosts => [host],
  :reliable => false,
  :max_reconnect_attempts => 1,
  :max_reconnect_delay => 1.0,
  :start_timeout => 1.0,
  :connect_timeout => 0,
  :connread_timeout => 1.0,
  :hbser => true,
  :stompconn => true
}
@conn = Stomp::Connection.new(hash)

I am not sure there is a way to accomplish what you want with 'poll'.  (At least as it is currently written.)  The socket was apparently 'not ready' (or already nil).
I think you would need to actually issue 'receive' (or 'disconnect') in order to see the exception.
Sorry.  I will think about this some more, and let you know if I come up with any further ideas.
Re: your connect hash ....... since you have reliable => false, those max_reconnect_xxxx parameters are ignored.  Hope you know that.


When i use both IPv4 and v6 address. How to force the v4/v6 traffic?

My config file is as follow:
# The [Interface] and [Peer] configurations follow the same semantics and meaning
# of a wg-quick configuration. To understand what these fields mean, please refer to:
# https://wiki.archlinux.org/title/WireGuard#Persistent_configuration
# https://www.wireguard.com/#simple-network-interface
[Interface]
Address = <IP_ADDRESS>/32
Address = 2606:4700:110:80de:c842:7e76:69e5:59e1/128 # The subnet should be /32 and /128 for IPv4 and v6 respectively
MTU = 1420
PrivateKey = cFQfJrir2UskpSlmyZgviLp3SWq5WVoGyESpd2s8q2M=
DNS = <IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>,2606:4700:4700::1111,2001:4860:4860::8888,2001:4860:4860::8844

[Peer]
PublicKey = bmXOC+F1FxEMF9dyiK2H5/1SUtzH0JuVo51h2wPfgyo=
# PresharedKey = UItQuvLsyh50ucXHfjF0bbR4IIpVBd74lwKc8uIPXXs= (optional)
Endpoint = <IP_ADDRESS>:1018
# PersistentKeepalive = 25 (optional)

.........

And my checking ip result is as follow. It seems the v4 / v6 is random
root@ubuntu:~# while [ "$i" != 10 ]; do curl -x socks5h://localhost:40000 -ksm5 -A Mozilla https://ifconfig.co/ip; ((i++));sleep 1; done
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
2a09:bac1:1120:18::61:289
2a09:bac1:1120:18::61:289
2a09:bac1:1120:18::61:289
2a09:bac1:1120:18::61:289
<IP_ADDRESS>
2a09:bac1:1120:18::61:289
<IP_ADDRESS>

root@ubuntu:~:~# unset i; while [ "$i" != 10 ]; do curl -x socks5h://localhost:40000 -ksm5 -A Mozilla https://api64.ipify.org; echo ''; ((i++)); done
2a09:bac1:1100:18::61:289
<IP_ADDRESS>
2a09:bac1:1100:18::61:289
2a09:bac1:1100:18::61:289
<IP_ADDRESS>
2a09:bac1:1100:18::61:289
2a09:bac1:1100:18::61:289
<IP_ADDRESS>
<IP_ADDRESS>
2a09:bac1:1100:18::61:289

I think ther is a bug in DNS. Please fix it . Thanks.


Traceback error when collecting information on disconnected vCenter host

Environment

Python version:  Python 3.6.9
NetBox version: v2.6.8
vCenter NetBox Sync version:  v1.0.0-rc

Steps to Reproduce

Disconnect a vCenter host
Execute run.py
Error gets thrown when collecting interface object info

Expected Behavior
Ignoring disconnected hosts

Observed Behavior
Traceback (most recent call last):
  File "run.py", line 1087, in <module>
    main()
  File "run.py", line 47, in main
    nb.sync_objects(vc_obj_type="hosts")
  File "run.py", line 791, in sync_objects
    vc_objects = self.vc.get_objects(vc_obj_type=vc_obj_type)
  File "run.py", line 348, in get_objects
    for pnic in obj.config.network.pnic:
AttributeError: 'NoneType' object has no attribute 'network'

That's a fantastic find! I'm looking into this. Thanks so much!

@dkuhnAtHome I perused through pyVmomi and I'm admittedly not in love with the http_client and session handler. I've added exception handling for these and a few other cases.
I was however not able to specifically replicate what you've done. When I go into vCenter and disconnect a host, vCenter still returns a cached instance of its data with the exception of status being returned as disconnected.
Would you mind telling me your vCenter version and whether you're doing anything different? If you're comfortable feel free to run the latest commit in develop branch (0881c142cde5ac2b7c88c8aac08f238094dcd553) and let me know if it's better. I'll hold on merging this in until I can test this further and ideally match your case 100%. Thanks!

I was able to reproduce and successfully saw exceptions being skipped and then picked up on the next run when available. I'll go ahead and merge this in. Please feel free to report back if you see otherwise. I'll leave it open for a few days.


Readme

What does this PR do?
Create a Youtube Clone
Description of Task to be completed?

Add basic structure
Create a Navigation bar
Create the main content
Create sidebar
Add footer
Add the README file.

How should this be manually tested?
Clone this repository
$ git clone https://github.com/paulzay/youtube
Go into the repository
$ cd youtube
Screenshots

Pull Request under Review by @santiago-rodrig (TSE)

Changes Required!
Hello @Haroonabdulrazaq and @paulzay, well done with adding your current changes to the correct PR. There are some changes we need to make to the project. These include;

[ ] Make sure that HTML passes the HTML validator tests.
[ ] The navigation should have at least three embedded images on the rightmost part of the navbar that represent icons and/or an avatar(Read more). I don't see them on your clone
[ ] The main video section should have a like and dislike icons(Read more).
[ ] The main video section should have a share button(Read more).
[ ] The main video section should show the avatar of the user who uploaded the video(Read more).
[ ] The main video section should have a subscribe button(not image). (Read more)

Kindly work on the required changes and submit another code review request
Feel free to contact me on slack if you need more info about this review
Treasure Kabareebe

Pull request currently under review by Michgolden Ukeje (TSE).
Kindly exercise patience 👋


typing.Union should not be joined if end of line has comments

black 18.6b4
from typing import Union

A = Union[
    int,  # my comment
    float,  # my other comment
]

gets translated into:
from typing import Union

A = Union[int, float]  # my comment  # my other comment

Interestingly black already handles lists well:
# this does not get reformatted
A = [
    int,  # my comment
    float,  # my other
]

FWIW, the list example also does get reformatted:
-A = [
-    int,  # my comment
-    float,  # my other
-]
+A = [int, float]  # my comment  # my other

Because all that code fits on one line including both comments, this is unlikely to get solved, except maybe for the type annotations case but the solution is unclear there as well: #282


HtmlEditorExtender drops spaces on postback of data pasted from word

The AJAX Control Toolkit  <IP_ADDRESS>
The installation method

[ ] Installer
[x] NuGet package
[ ] A custom build from the source code

Minimal steps to reproduce the bug
<asp:TextBox ID="txtMessage" runat="server" CssClass="" Height="150px" />
<cc2:HtmlEditorExtender ID="txtMessageExt" TargetControlID="txtMessage" runat="server" DisplayPreviewTab="true" DisplaySourceTab="true"  />

Consider a page with that arrangement of TextBox and HtmlEditorExtender
Pasting the following text typed into Word:
Matthew and Kirbie,
Congratulations on the successful agreement you have secured to sell your home.
My name is Jamie Dickson with Berkshire Hathaway Homes Services – The Preferred Realty. I am the transaction coordinator for Joanne Bates and will be assisting in managing your transaction file. As you will be receiving documents, email communication and information from me occasionally at Joanne’s request, I wanted to take this opportunity to introduce myself.
You will be receiving copies of all of your listing contracts and additional documents by email, as well as a link to your documents. Those will be sent in a separate email.
I am looking forward to participating in providing you with quality service.

Actual result
On postback, the data is bungled to:
Matthew and Kirbie,

Congratulationson the successful agreement you have secured to sell your home.

Myname is Jamie Dickson with BerkshireHathaway HomesServices - The Preferred Realty.  I am the transactioncoordinator for Joanne Bates and will be assisting in managingyour transaction file.  As you will be receiving documents,email communication and information from me occasionally at Joanne’s request, I wanted to take this opportunity tointroduce myself.

Youwill be receiving copies of all of your listing contracts and additionaldocuments by email, as well as a link to your documents.  Those will be sent in a separate email.

I amlooking forward to participating in providing you with quality service.

Expected result
The text to not drop spaces pasted into the editor
Browser(s) used
Chrome 58, Chrome 60, Internet Explorer 11
A site deployment method

[x] VS development web-server,
[ ] IIS

The same issue is present with HtmlEditor starting in at least version 8 of the Nuget package.

Hi @jesanfafon
Do you see newlines before the postback, when you just have pasted the text?

No, I do not.
Here's the exact value upon post-back:

Matthew and Kirbie,

Congratulations on the successful agreement you have securedto sell your home.

My name is Jamie Dickson with Berkshire Hathaway HomeServices – The Preferred Realty. I am the transaction coordinator for JoanneBates and will be assisting in managing your transaction file. As you will bereceiving documents, email communications and information from me occasionallyat Joanne’s request, I wanted to take this opportunity to introduce myself.

You will be receiving copies of all of your listingcontracts and additional documents my email, as well as a link to yourdocuments. Those will be sent in a separate email.

I am looking forward to participating in providing you with qualityservice.

Seems to me from some minor investigation, that it might have something to do with normalizing line endings.

@jesanfafon I can not reproduce this issue so that HtmlEditorExtender behaves exactly as your case.
To narrow down the problem, please check my steps:

I copy sample text into Word 2016:

Please note that Word creates new paragraphs on each new line which can easily be seen.

I copy this text from Word into a content editable div I created at jsfiddle using Chrome 58:

HtmlEditorExtender uses div with a contenteditable attribute set to "true", so there is no difference.
As you can see, Word paragraphs are converted into HTML paragraphs, as expected.

A similar result occurs when I paste the text using IE:

Now I paste same text as a plain text using Chrome:

The result is different:

As you can see, there is no HTML paragraph. Text is wrapped into divs instead.

So far, it seems to be a browser-dependent rendering issue which affects contenteditable divs.
Please inform us of your results with fiddle, so we can investigate it further.

Yes, that all makes sense! To be clear, the issue I'm having isn't that the text looks wrong in the editor after pasting (to the contrary, it looks perfect), it's that the data being posted by the control has the spaces dropped:

@jesanfafon I am still not sure what dropped spaces you mentioned.
Would you please attach a sample Word file to the issue and indicate what parts of this document are affected?

Sure can.
If you look at the image I attached, you'll see that there are periodically spaces missing between words that have spaces between them in the word document (and on screen in the actual control).
If you inspect the data being posted back from the control, you will see that the value in the textarea has spaces missing compared to the word document.

Matthew and Kirbie,

Congratulations on the successful agreement you have securedto sell your home.

My name is Jamie Dickson with Berkshire Hathaway HomeServices – The Preferred Realty. I am the transaction coordinator for JoanneBates and will be assisting in managing your transaction file. As you will bereceiving documents, email communications and information from me occasionallyat Joanne’s request, I wanted to take this opportunity to introduce myself.

You will be receiving copies of all of your listingcontracts and additional documents my email, as well as a link to yourdocuments. Those will be sent in a separate email.

I am looking forward to participating in providing you with qualityservice.

CopySource.docx

I was looking into this a bit more yesterday, and the value in the backing input has the spaces dropped immediately after pasting the word doc content into the editable div.

@jesanfafonI I see that the space between the listing contracts disappeared, but cannot reproduce this issue.
Would you please record a video to illustrate it? You can use the Jing tool for this.

Since we still haven't received the requested information, I'm closing this issue.
Feel free to reopen it once you have any additional info regarding the error.

thanks
Sent from my HUAWEI NXT-AL10 using FastHub

Since we still haven't received the requested information, I'm closing this issue.
Feel free to reopen it once you have any additional info regarding the error.

I just ran into this issue.  I put together a quicky page for my boss to send out mailings for his golfing group.  He sent me a word doc, I copy+pasted it into an HTMLEditor  I was shocked when I got an angry "what did you do to my text?" that I'm trying to track down.
I opened this with Word 2007
PropertyOwnersLetter3.docx
I did a "Select All" and copy and then did a paste into my
<asp:TextBox ID="HTMLEdit" runat="server" Height="600" Width="100%" />
                                <ajaxToolkit:HtmlEditorExtender ID="hHTMLEdit" runat="server" TargetControlID="HTMLEdit" EnableSanitization="false" DisplaySourceTab="true" />

And visually, it looks like this(added red box to point out where my first error eventually comes in)

Switching to source view it still looks fine

If I open up developer tools, I can see some strangeness, but, you'll see in the word doc there's no newline, only a space(which is how it renders onscreen)

And does not come over correctly on postback

That was with Edge(New) 84.0.508.0
Same results with Edge(Old) 44.19624.1000.0

FireFox 78.0.a1 looks a bit different

But ends up destroying the space between the "to" and "my" just like the other two browsers(sorry, since so much header was retained, had to paste into notepad to see).

Meanwhile, I paste into notepad, and it's fine

If I dump a second multiline textbox, it's also fine

So is this a windows pasting problem?