MeiliSearch::TimeoutError

Description
After I send a search request with the following params (below), I get MeiliSearch::TimeoutError from both meilisearch rails and ruby libraries.
{
  "offset": 0,
  "limit": 5,
  "attributesToHighlight": [
    "title",
    "description",
    "authors_simple"
  ],
  "filter": "territories=US AND (language=en)",
  "q": "I will always write be"
}

This problem repeats for any q value which has its length bigger than 21 characters, including spaces.
Expected behavior
Search should return a proper result.
Current behavior
/home/deployer/proxy_search/shared/bundle/ruby/3.1.0/gems/meilisearch-0.22.0/lib/meilisearch/http_request.rb:113:in `rescue in send_request': The request was not processed in the expected time. Net::ReadTimeout with #<TCPSocket:(closed)> (MeiliSearch::TimeoutError)
/home/deployer/proxy_search/shared/bundle/ruby/3.1.0/gems/net-protocol-0.2.1/lib/net/protocol.rb:229:in `rbuf_fill': Net::ReadTimeout with #<TCPSocket:(closed)> (Net::ReadTimeout)

Environment (please complete the following information):

OS: Ubuntu 20.04
Meilisearch version: 1.1.0
meilisearch-ruby version: v0.22.0

Sorry for the late reply, here're the settings and some common objects we have.

Settings

 =>
{"displayedAttributes"=>["*"],
 "searchableAttributes"=>["title", "authors_simple", "categories", "keywords", "publisher", "narrators_names"],
 "filterableAttributes"=>
  ["anyplay_book_id", "authors_simple", "cache_status", "language", "licenses_count", "narrators_names", "publisher", "territories", "title"],
 "sortableAttributes"=>["rating_int:desc", "release_date_int:desc"],
 "rankingRules"=>["words", "typo", "licenses_count:desc", "proximity", "attribute", "sort", "exactness"],
 "stopWords"=>[],
 "synonyms"=>{"nyc"=>["new york"]},
 "distinctAttribute"=>nil,
 "typoTolerance"=>{"enabled"=>true, "minWordSizeForTypos"=>{"oneTypo"=>2, "twoTypos"=>4}, "disableOnWords"=>[], "disableOnAttributes"=>[]},
 "faceting"=>{"maxValuesPerFacet"=>2000},
 "pagination"=>{"maxTotalHits"=>1000}}

A common response object

{"hits"=>
  [{"title"=>"Dead Silence",
    "authors_simple"=>"S.A. Barnes",
    "categories"=>["Action & Adventure", "Science Fiction", "Horror", "Fiction"],
    "keywords"=>"[\"Action & Adventure\", \"Science Fiction\", \"Horror\", \"Fiction\"]",
    "narrators"=>["Lauren Ezzo"],
    "anyplay_book_id"=>546351,
    "id"=>"546351",
    "book_id"=>"449216",
    "cover_url"=>"http://images.findawayworld.com/v1/image/cover/CD433430",
    "description"=>
     "
\"The audiobook read by Lauren Ezzo makes for compulsive, edge-of-seat listening.\" - Buzzfeed

\"Ezzo's dark tone and emotional connection terrify and comfort listeners in equal measure. Not for the faint of heart.\"-AudioFile Magazine

Titanic meets The Shining in S.A. Barnes’ Dead Silence, a SF horror novel in which a woman and her crew board a decades-lost luxury cruiser and find the wreckage of a nightmare that hasn't yet ended.

A GHOST SHIP.
A SALVAGE CREW.
UNSPEAKABLE HORRORS.

Claire Kovalik is days away from being unemployed—made obsolete—when her beacon repair crew picks up a strange distress signal. With nothing to lose and no desire to return to Earth, Claire and her team decide to investigate.

What they find at the other end of the signal is a shock: the Aurora, a famous luxury space-liner that vanished on its maiden tour of the solar system more than twenty years ago. A salvage claim like this could set Claire and her crew up for life. But a quick trip through the Aurora reveals something isn’t right.

Whispers in the dark. Flickers of movement. Words scrawled in blood. Claire must fight to hold onto her sanity and find out what really happened on the Aurora, before she and her crew meet the same ghastly fate.

A Macmillan Audio production from Tor Nightfire
",
    "rating"=>4.33,
    "release_date"=>"2022-02-08",
    "slug"=>"dead-silence-0aa1e265-81d9-4240-ae1a-bae4f1380376",
    "cache_status"=>"cached",
    "territories"=>
     ["AD",
      "AE",
      "AF",
      "AG",
      <...>,
      "ZW"],
    "language"=>["en"],
    "created_at"=>"2023-04-06T16:52:51.502Z",
    "updated_at"=>"2023-04-06T16:52:51.502Z",
    "rating_int"=>43,
    "release_date_int"=>2022,
    "licenses_count"=>141},
   {"title"=>"They Stole Our Hearts",
    "authors_simple"=>"Daniel Kraus",
    "categories"=>["Toys, Dolls, Puppets", "Horror & Ghost Stories", "Bears", "Animals", "Juvenile fiction"],
    "keywords"=>"[\"Toys, Dolls, Puppets\", \"Horror & Ghost Stories\", \"Bears\", \"Animals\", \"Juvenile fiction\"]",
    "narrators"=>["Kirby Heyborne"],
    "anyplay_book_id"=>477578,
    "id"=>"477578",
    "book_id"=>"417119",
    "cover_url"=>"http://images.findawayworld.com/v1/image/cover/CD401918",
    "description"=>
     "
The heart-pounding sequel to They Threw Us Away, about a group of teddy bears looking for a place to call home…and answers to life’s biggest questions.

The teddies—clever Buddy, brave Sunny, sweet Sugar, and wise Reginald—have managed to find a child. Life with Darling is far better than any they’ve known. But something's not right—the promised bliss of Forever Sleep hasn’t come. And they are kept a secret from Darling’s mother, hidden underneath the child’s bed in the dusty darkness.

Then the inevitable happens: Mama discovers the teddies. And like all adults they’ve met thus far, she responds with fear and anger. The teddies must watch as one of their friends is destroyed. The remaining trio barely escape, thrust back into a world that does not want them.

Disillusioned and lost, the teddies embark on a journey back to the factory where they were created. En route, they find a civilization of discarded teddy bears. The comfort of a town of teddies has its allure…but the need for answers weighs heavy. And there’s something definitely off about these new teddies. Will our heroes accept their strange rules? Or must they dig deep for one more grand adventure to finally learn why they were thrown away?
",
    "rating"=>3.5,
    "release_date"=>"2022-01-11",
    "slug"=>"they-stole-our-hearts",
    "cache_status"=>"cached",
    "territories"=>
     ["AD",
      <...>,
      "ZW"],
    "language"=>["en"],
    "created_at"=>"2023-04-06T20:11:42.873Z",
    "updated_at"=>"2023-04-06T20:11:42.873Z",
    "rating_int"=>35,
    "release_date_int"=>2022,
    "licenses_count"=>123}
]

Just in case, territories is an array that may have up to 255 territories.  I tried reproducing the issue using the search query, settings, and the two sample documents you provided. However, I was unsuccessful in reproducing the problem through a direct call to the engine using cURL and Postman. Can you confirm if you were able to reproduce the issue with the same set of data you provided? If not, could you please share the exact documents you used and the steps you took to reproduce the issue using a direct call to the engine? Thanks

I'll close this issue out as it has been a while since it's been updated. Please feel free to re-open if there's additional information available to continue 🙏🏻


Next page from URL instead of pagination?

Is it possible to set path for selector that only contains one link and then load, i am trying to make infinite scroll in wordpress posts that will load next post but cant have pagination there, i tried something but its too hacky and wont update history url correctly.
Any help?

Is it possible to set path for selector that only contains one link and then load, i am trying to make infinite scroll in wordpress posts that will load next post but cant have pagination there, i tried something but its too hacky and wont update history url correctly.
Any help?

sj

@desandro @sj5505 Did you get this fixed? I'm getting errors when trying to load next and previous posts. Works fine on archives but not on single posts.

i'm having the same issue. Actually it auto updates the url of next post with a change in post date or adding a different number at the end of url which leads it to 404. I don't know what's the matter with this.

Did anyone found a workaround for this? Looking to implement the same use case as @magix01, where I want to load the next post from a normal WordPress post url.
I've tried to set path as function to then query the next post link element, but that doesn't work and will always load the second post in the sequence over and over again.

Sounds like you're asking for Loading URLs from loaded pages. Previously asked for in #688. Report back if you're looking for different kind of behavior.


3D Commerce Certification Test

Copy the 3D commerce certification models into test/certification/models for the test to work. This is a WIP, but it's good enough to create the zip for certification.
https://user-images.githubusercontent.com/19158740/120871868-c8fd5300-c551-11eb-8eae-b17977293548.mp4

This is ready for review. I've added some instructions to the folder to explain how to run it.

Do we want to run it in our CI?

Do we want to run it in our CI?

Not yet. This test doesn't currently pass given the same threshold as the validation tests. We are waiting for Khronos to come up with the correct criteria for the tests. We have already submitted something and will wait to see what they say.


Add Durable property to ConsumerOptions and QueueOptions

Feature description

Namespace ActiveMQ.Artemis.Client.Extensions.DependencyInjection.
ConsumerConfiguration class has Durable property.
QueueConfiguration class has Durable property.
ConsumerOptions and QueueOptions don't have Durable property and we are not able to set this property when we call AddConsumer() method.
Add Durable property mapping to ConsumerConfiguration and QueueConfiguration in AddConsumer() method.

Feature in action
public static IActiveMqBuilder AddConsumer(this IActiveMqBuilder builder, string address, RoutingType routingType, string queue, ConsumerOptions consumerOptions, QueueOptions queueOptions,
            Func<Message, IConsumer, IServiceProvider, CancellationToken, Task> handler)
        {
            builder.Services.Configure<ActiveMqOptions>(builder.Name, options =>
            {
                options.QueueConfigurations.Add(new QueueConfiguration
                {
                    Address = address,
                    RoutingType = routingType,
                    Name = queue,
                    Exclusive = queueOptions.Exclusive,
                    FilterExpression = queueOptions.FilterExpression,
                    GroupBuckets = queueOptions.GroupBuckets,
                    GroupRebalance = queueOptions.GroupRebalance,
                    MaxConsumers = queueOptions.MaxConsumers,
                    AutoCreateAddress = queueOptions.AutoCreateAddress,
                    PurgeOnNoConsumers = queueOptions.PurgeOnNoConsumers,
                    LastValueKey = queueOptions.LastValueKey,
                    NonDestructive = queueOptions.NonDestructive,
                    // *** Add this line ***
                    Durable = queueOptions.Durable
                });
                if (options.AddressConfigurations.TryGetValue(address, out var routingTypes))
                {
                    routingTypes.Add(routingType);
                }
                else
                {
                    options.AddressConfigurations[address] = new HashSet<RoutingType> { routingType };
                }
            });
            for (int i = 0; i < consumerOptions.ConcurrentConsumers; i++)
            {
                builder.AddConsumer(new ConsumerConfiguration
                {
                    Address = address,
                    Queue = queue,
                    Credit = consumerOptions.Credit,
                    FilterExpression = consumerOptions.FilterExpression,
                    NoLocalFilter = consumerOptions.NoLocalFilter,
                    // *** Add this line ***
                    Durable = consumerOptions.Durable
                }, observable =>
                {
                    observable.Address = address;
                    observable.RoutingType = routingType;
                    observable.Queue = queue;
                }, handler);
            }

            return builder;
        }

Describe alternatives you've considered
Additional context

Hi @babinecm,
Each queue created with management API will behave as durable. There is no easy way to change this.
What is your use case?
Thanks,
Havret

Can you confirm that EnableQueueDeclaration and EnableAddressDeclaration are included in your app setup, similar to the example found in the sample app?

Artemis automatically adds a 'global' postfix when you create a subscription using AddSharedDurableConsumer. To disable this, you need to modify the broker configuration as demonstrated in the broker.xml.

I hope that helps,
Havret

Thank you for your suggestion.

Unfortunately if I use EnableQueueDeclaration or EnableAddressDeclaration or both, I get error and the client won't connect at all.

ArtemisNetClient: Failed to send the message
Cannot invoke "org.apache.activeng.artemis.core. transaction.Transaction.markAsRollbackOnly (org.apache.activemg.artemis.api.core.ActiveMException)" because "x" is null

at ActiveM.Artemis.Client.RequestReplyClient.<SendAsync>d__12.MoveNext()
at ActiveM.Artemis.Client.AutoRecovering.AutoRecoveringRequestReplyClient.<SendAsync>d__7.MoveNext()
at ActiveM.Artemis.Client.TopologyManager.<SendAsync>d__20.MoveNext()
at ActiveM.Artemis.Client.TopologyManager.<GetQueueNamesAsync>d__9.MoveNext()
at ActiveM.Artemis.Client.Extensions.DependencyInjection.ActiveMqTopologyManager.<CreateTopologyAsync>d__6.MoveNext()
at ActiveM.Artemis.Client.Extensions.DependencyInjection.ActiveMqTopologyManager.<CreateTopologyAsync>d__6.MoveNext()
at ActiveM.Artemis.Client.Extensions.DependencyInjection.ActiveMqClient.<StartAsync>d__5.MoveNext()

I can't do that. I don't have access to this file. Only another team in our organization can do this. I'll try.

What version of ActiveMQ Artemis are you on? Maybe your app doesn't have access to the activemq.management queue? You need this for the TopologyManager to set up addresses and queues when it starts, based on your config.

If you can get this changed, just using AddSharedDurableConsumer ought to do the trick for what you're trying to do."

We have ActiveMQ Artemis v2.20.0

I tried to update ActiveMQ.Artemis.Client.Builders.ConsumerBuilder class, just as a proof of concept.
...
private static Symbol[] GetCapabilities(ConsumerConfiguration configuration) => new[] { Capabilities.Multicast };
...
private static uint GetTerminusDurability(ConsumerConfiguration configuration) => TerminusDurability.UnsettledState;

private static string GetReceiverName(ConsumerConfiguration configuration) => configuration switch
{
    { Shared : true } => configuration.Queue,
    { Queue : not null and not "" } => configuration.Queue,
    _ => Guid.NewGuid().ToString()
};

I called AddConsumer("event-topic", RoutingType.Multicast, "my-queue", handler) and it worked. I can now see "my-queue" as multicast in Artemis browser UI and is also durable.
So, how to achieve this with the consumer configuration?
I've tried many configurations but nothing works except the "hard edit" described above.

I've tried nuget v2.15.0-preview1 and the routing type now works. Thank you for quick response.
But queue is not durable.
Can you please add Durable = consumerOptions.Durable in the same place as you added the RoutingType = routingType?
Default behavior won't change and we can be able to set the consumer as durable as I did here
Durability is determined by ConsumerConfiguration in ConsumerBuilder class:
private static uint GetTerminusDurability(ConsumerConfiguration configuration) => configuration switch
{
    { Durable: true } => TerminusDurability.UnsettledState,
    _ => TerminusDurability.None
};

+1 We are also looking for an option to make consumers durable using the consumer options.
Kind regards,
Jacob

I've just run a quick check against Artemis v2.20.0, and the queue appears to have been created as durable. I know this is a minor change to make, but I would prefer to make it only if necessary. I would need to rethink the API since ConsumerOptions is also used for AddSharedConsumer and AddSharedDurableConsumer. Having these options available there wouldn't make much sense.

If you don't want to change ConsumerOptions, then you can add a new AddDurableConsumer extension method to ActiveMqArtemisClientDependencyInjectionExtensions.cs. Here you can set Durable = true.
Is this acceptable to you?

However, the question remains: do we really need it? As I mentioned in the comment above, I've verified that in the current build against Artemis v2.20.0, the queue is created as durable without the need to change TerminusDurability.

I see, but it doesn't work in our environment. Maybe it's some configuration on the server, but I don't have access to the server configuration. I can only see the list of available queues.
I did the following tests:

Nuget 2.15.0-preview2. Connect to the server. The queue was created. Disconnect from the server. The queue disappeared from the server after 10 min.
Modified nuget 2.15.0-preview2, I only added Durable = true as described above. Connect to the server. The queue was created. Disconnect from the server. The queue remained on the server and is still there.

For some reason I have to explicitly tell the server to create a durable queue, otherwise the queue will be deleted after some time.
I will really appreciate it if you add the option to explicitly define the queue durability.

It would be worthwhile to check your server configuration. Could you reach out to your Artemis Administrator and ask them to share the broker.xml file you guys are using?
There are a few settings that control when Artemis may delete queues and addresses:
    <auto-delete-queues>true</auto-delete-queues>
    <auto-delete-addresses>true</auto-delete-addresses>
    <auto-delete-queues-message-count>0</auto-delete-queues-message-count>
    <auto-delete-queues-delay>1000</auto-delete-queues-delay>

If I apply this configuration to my local instance of Artemis, the broker will remove the queue regardless of the change you're suggesting (Durable = true).
That being said, I'd prefer to get to the root cause of why the client and broker behave as they do, rather than making haphazard changes to the library that may seem to work but could blow up at the most unexpected moment in production because our understanding of the root cause was too superficial.
I hope that makes sense.

Here is the response from our Artemis Administrator.

The current version of Artemis MQ is 2.31.2.
With the restart of brokers, it deletes all addresses that are not saved in the configuration and no app is connected to the given address.
Client connection to management API is disabled.

Configuration from broker.xml:

<auto-create-queues>true</auto-create-queues>
<auto-create-addresses>true</auto-create-addresses>
<auto-create-jms-queues>true</auto-create-jms-queues>
<auto-create-jms-topics>true</auto-create-jms-topics>
<config-delete-queues>FORCE</config-delete-queues>
<config-delete-addresses>FORCE</config-delete-addresses>

Thanks for getting back to me with the config. From what I can see, the problem lies in the missing options:
  <auto-delete-queues>false</auto-delete-queues>
  <auto-delete-addresses>false</auto-delete-addresses>
  <auto-delete-created-queues>false</auto-delete-created-queues>

These are set to true by default. So, your addresses and queues will be automatically removed if there are no attached consumers, producers, or if there are no unconsumed messages in the queues. My guess is that the latter was giving you the impression that sometimes the queue is not removed and sometimes it is. It was not driven by the client setting but by the outstanding messages in the queue. The settings config-delete-queues and config-delete-addresses deal only with config-created addresses/queues, meaning queues and addresses that are explicitly defined in your broker.xml file. For instance, you can define a queue in the following way:
 <address name="test-address">
    <anycast>
       <queue name="test-queue">
          <durable>true</durable>
       </queue>
    </anycast>
 </address>

If you remove it from the config file, it won't be automatically removed from your broker topology unless you explicitly set <config-delete-queues>FORCE</config-delete-queues>. The same applies to addresses.
In summary, please ask your Artemis Administrator to disable the auto-delete queues and addresses options, and everything should work fine (no queues should be removed). Another option is to ask them to explicitly define your queues as part of the broker configuration.
I hope that helps,
Havret

Thanks for the help and explanation. I don't know how it works in detail. Now I see that the problem is in the server configuration. I'll try to ask our Artemis Administrator if this behavior can be changed.
At least, we fixed the routing type when using FQQN #472.
Thanks again. So I'm closing this issue.


The introduction of the endpoints page is referencing to the wrong product

Hello,
I was reading about this endpoint and found this section 'These include things like a group mailbox for conversations and a group OneDrive folder for documents and files.'
OneDrive to my understanding is a service connected with a individual and not a Microsoft 365 Group. I assume you are mean SharePoint in this case?
Thank you for your feedback,
Patrick

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cb18386c-dc40-87d2-a200-1e1f314ecd3f
Version Independent ID: 931e3c54-1276-3b3a-f3bd-7ecf104ba5e4
Content: Endpoint resource type - Microsoft Graph beta
Content Source: api-reference/beta/resources/endpoint.md
Product: ****
Technology: microsoft-graph
GitHub Login: @yyuank
Microsoft Alias: MSGraphDocsVteam

As part of a repository clean up effort we are closing older issues. If this issue remains, please: open a new issue, reference this issue, and provide any additional details that may help in resolution. Thank you for your understanding as we work to improve our responsiveness.


Demag GUI help

The Demag GUI help message isn't super useful for someone trying to run Demag GUI on the command line.  Maybe we should point them to the Cookbook, or provide information about the command line options?

Fair enough I've focused so much on the in context help I kinda forgot this was a thing; I'll update the doc string tomorrow or Monday so it's more standardized, informative, and includes links to further sources.

Make sure to tag this commit for the docathon (https://bids.github.io/docathon/):

Any documentation commits should include the words "doc", "documentation", or "docathon" in the commit message. During the week, we'll be keeping track of stats on who's contributed. This will be displayed on our lovely stats page. Below are the current leaders in both projects and users. We'll update these plots a few times each day so that we can congratulate those that have made contributions. You can also click a plot to go to a summary page.

Hey @ltauxe and @ronshaar in my update of the Demag GUI documentation I want to include a script style docstring which describes how to run from command line and should go where the "Log Header" is currently. For now I'm just going to move the Log Header down and put in the docstring, but this Log Header is defiantly out of date, seems redundant considering we use github now, and uses up a decent bit of space in the script. So I would like your opinions on removing this Log Header from the top of Demag GUI and possibly Thellier GUI and instead just including a list of authors at the bottom of the docstring. Thanks.

The new help looks good to me!

K, I'm going to close this for now then so it's not cluttering the issues page.


Action row floating run button sometimes is offset to the row

To Reproduce
Steps to reproduce the behavior:

Select action in the grid
Resize the row or column size
action floating buttons placed incorrectly on wrong row/col

Expected behavior
Actions floating buttons to be in the row boundreis
Screenshots

New Automation page will be ready soon, so obsolete


Update default data ocean for v12?

Recently, @AgilentGCMS discovered that the Reynolds o1 ocean seems to...stop in 2022. If you try running in 2023+ it crashes. I've asked @rcullath to see if this is expected or not, but I think this brings up another question, mainly for @wmputman but also @sdrabenh: Should we change the default data ocean in v12?
At the moment in v12 (and v11), the default data ocean is just Reynolds; from C12 to C5760, it's Reynolds. This seems...dumb. If nothing else, we should probably default to Ostia if we want to default to a lat-lon (I think Ops uses it?) and maybe even CS ocean?
I think we have good remapped ocean data of C90 up for the "usual" uniform grids. The ultra-high-res and stretched are in sort of a worse shape but we can make those if needed.
I could work with @sshakoor1 to fix up the scripts pretty easily to make the change. But just need guidance.

Also something @AgilentGCMS reminded me of. Whatever we choose here for a default should also be what remap_restarts.py does. It looks like it might choose CS as the default to remap to, so we should probably echo that here.
A question for @weiyuan-jiang would be, is it possible for that script to default to:

CS ocean if C90+
Reynolds or MERRA2 if lower?

Also something @AgilentGCMS reminded me of. Whatever we choose here for a default should also be what remap_restarts.py does. It looks like it might choose CS as the default to remap to, so we should probably echo that here.
A question for @weiyuan-jiang would be, is it possible for that script to default to:

CS ocean if C90+
Reynolds or MERRA2 if lower?

Yes, I think it is not hard

The script already did what you want:
default_ = 'CS  (same as atmosphere OSTIA cubed-sphere grid)'
if resolution in ['C12','C24', 'C48'] : default_ = '360x180   (Reynolds)'


Request to list all missing crates rather than first one

Request to list all missing crates rather than exit on the first one.
A bespoke but not entirely unlikely use case:
As a company that has locally cached crates in a dir via some bespoke process. when the source code is updated and depends on different crate versions we run cargo check and get a the first missing crate/version. Then we repackage (somehow) to include that crate version, then we repeat until we have them all.
It would be nice if cargo listed all the missing crates/versions it needed but couldn't find before it exited.

I am not entirely sure I know which error you are referring to. Do you mean the case of
[dependencies]
not-a-thing="*"
not-another-thing="*"

whos error only menchens one of them?
In general listing all the transitive deps that are not available is hard. Even just proving that there is no way around any one problem is np-complete. So listing all of them may be a lot of work. But, it seems reasonable to add a bespoke pass over the direct dependencies, before resolution. Is it is worth the code complexity? Depends on implementation details, that I can't remember at the moment.

Yes that's right. If it could tell me all the immediate dependencies it's missing that would be good. At the moment, it's one by one that I'm filling in blanks. If it could tell me one level at a time that would significantly cut down the number of runs I need to do to find out the information.
This is an example error:
error: failed to select a version for the requirement `chrono-tz = "=0.5.2"`
  candidate versions found which didn't match: 0.5.3


fix media type validation

if the Content-Type is "application/json;charset=utf-8" validation fails.

Hi @marimelon
the fix is released in version 0.7.1. Thank you for contribution.
Jirka


Implement/check functioning of the Dictofun functions in the background

Put the android app to the background (or make it a background service) and test if it works after 4-8-24 hours being inactive.
If it doesn't - implement background operation

It works almost OOB. No actions needed


Create F1 dataset link pred task

Create the F1 dataset link pred task which is to predict a list of constructors a driver will join in the next 10 years, according to F1 race results.
This seems to be one of few link pred task we can create. Tables having timestamp include results, qualifying, standings and races. Following is the schema of the database

Previously @rusty1s proposed to make a link pred task: for each race predict top k drivers. But race actually has timestamp and it can't be set as src entities because our current link pred task doesn't support prediction with new entities (dangling filtered)
So the proposed link pred task is to predict a list of constructors a driver will join in the next 10 years, according to F1 race results. The number of entities and degree information is shown below (1/1/2002 is for the val table and 1/1/2013 is for the test table)

Timestamps
Total # src nodes
Total # dst nodes
Avg Degree

1/14/1952
341
634
1.8592

1/11/1962
172
402
2.3372

1/9/1972
171
407
2.3801

1/6/1982
111
257
2.3153

1/4/1992
98
200
2.0408

1/1/2002
73
142
1.9452

1/1/2013
57
105
1.8421

Model performance:
Here is the formatted markdown table:

Model
Split
Link Prediction Precision
Link Prediction Recall
Link Prediction MAP

gnn_link
Validation
0.049315
0.093836
0.063090

gnn_link
Test
0.073684
0.244737
0.109254

idgnn
Validation
0.038356
0.099087
0.097717

idgnn
Test
0.056140
0.151170
0.149220

past_visit
Validation
0.038356
0.099087
0.078995

past_visit
Test
0.038596
0.111111
0.067836

global_popularity
Validation
0.019178
0.044977
0.044977

global_popularity
Test
0.021053
0.039474
0.039474

@weihua916 Thanks, just updated


[DNM] [RFC] drivers: i2c: Add i2c_async for asynchronous transfers

PR content:

Proposed extension to i2c API:

It's is mainly addition of i2c_single_transfer(dev, msg, add, callback, user_data).
Extending i2c_msg flags to contain milliseconds delay field which can be used within sequence to delay message execution.
Added helper macros for initializing set of message, e.g. I2C_MSG_TXRX()
Added function for getting i2c_async instance associated with given device: i2c_get_async(dev)

Added new module drivers/i2c/i2c_async which wraps queue_operation_manager and async_seq_manager to provide safe and managed, asynchronous access to i2c.

Module is a fairly simple wrapper which binds those two modules together and finally uses i2c_single_transfer() to execute item within the sequence.
I2C sequence may consist of many i2c_async_msg items and may be constant, e.g. whole sensor initialization procedure can be stored in flash.
There are 3 message types:

transfer which contains struct i2c_msg
delay - sequence is delayed for milliseconds
pause - user provides function which is called when message is executed. Sequence is resumed using i2c_async_resume().

i2c_async is a generic part and vendor specific driver implements only single transfer functionality (along with configure and getting async instance). This way specific driver is only focused on it's core functionality, all access management, semaphores, etc. is removed from the driver. It is in opposition to current state where each driver is implementing own mechanisms for access control.
i2c_async implements z_i2c_async_sync_transfer which is used to provide backward compatibility (it is called by i2c_transfer).

Modified i2c_nrfx_twim driver to implement i2c_single_transfer

Modified hts221 sensor driver to show how to use i2c_async. This is temporary of course, just to show new way.

How to control I2C using i2c_async?
Asynchronous access to i2c does not happen through i2c API because there can be multiple context accessing same i2c device. Access must be managed. queued operation manager is created to handle that generic task (see #23333) thus user is actually talking to queued operation manager API using handle to the manager associated with given i2c device. The manager works with queued operations object so i2c operation must contain queued operation structure. i2c operation consists of one or more i2c_msg. Messages build a sequence which is executed by the queue operation manager. Completion of the i2c operation is notified using async notify object (see #23229) which is part of the i2c operation. async notify standardizes methods of notification of completion of asynchronous operation.
Scheduling of asynchronous i2c is performed in following steps:

create i2c operation (see struct i2c_async_op in i2c_async.h). Helper macros can be used to create it. Operation contains queued operation object and pointer to the sequence (pointer because sequence can be in flash and operation must be in rw memory).
get i2c_async object to use. Use i2c_get_async(dev).
initialize async_notify. Use i2c_async_get_notify(op) to get async notify from the operation.
submit operation queue_operation_submit(mngr, op, PRI)
wait for notification about the completion

Example code:
/* create an operation with sequence */
I2C_ASYNC_OP_DEFINE(op, I2C_SENSOR_ADDRESS,
		I2C_ASYNC_MSG_TXRX(&reg_who_i_am, 1, &who_i_am, 1),
		I2C_ASYNC_MSG_PAUSE(who_i_am_validate),
		I2C_ASYNC_MSG_TX(reg_ctrl1, 2),
		I2C_ASYNC_MSG_DELAY(3),
		I2C_ASYNC_MSG_TXRX(&reg_conv, 1, conv_data, sizeof(conv_data))
	);

/* async instance */
struct i2c_async *i2c_async = i2c_get_async(i2c_dev);
struct async_notify *anotify = i2c_async_get_notify(&op);

/* get callback notification */
async_notify_init_callback(anotify, i2c_callback);

/* submit operation */
err = queued_operation_submit(i2c_async_get_qop_mgr(i2c_async), &op.op,
					QUEUED_OPERATION_PRIORITY_APPEND);
if (err < 0) {
	return err;
}

Benefits

operations can be submitted from interrupt context
backward compatibility
driver code focused on its functionality (single transfer)
no need to extend i2c API with generic API (queued operation manager API used directly)
similar approach can be applied to spi
standardized asynchronous notification (async_notify)
possibility to combine mulitple i2c operations into a sequence (e.g. init)
possibility to pause a sequence (not added yet to i2c_async) which allows to combine operation like trigger sensor, wait for data rdy pin, read sensor into a single sequence.
possibility to have single sensor implementation for spi or i2c bus with wrapper for creating messages (e.g. SENSOR_BUS_MSG_TXRX(), SENSOR_BUS_OP_DEFINE)

misc
PR is based on unmerged work:

async_notify #23229
queued_operation #23333
async_seq #23075

hts221 sensor changes commit will be removed from final PR

Extending i2c_msg flags to contain milliseconds delay field which can be used within sequence to delay message execution.

I'm not a fan of packing up many features into a single API that would have be be independently supported in multiple drivers.  Why should a single-transaction API include a start delay?
Better to have a multi-operation description that steps through a sequence of operations, one of which could be a generic "delay" (for time, or for event) that is interpreted by an service-agnostic multi-operation sequencing engine, while service-specific operations are passed off to a common driver function like process_operation.

@tbursztyka

The example is maybe not the best one (at least the init process: it looses the error check on chip id afaik)

yes, it might be that chip id would need to be taken out, alternatively it can be checked after whole sequence. Another option that sequence manager is supporting is pausing. sequence can be configured so that after chip id read there is a pause callback. User checks chip id and aborts sequence if it is wrong or resumes. This is not yet implemented in i2c_async (supported by async_seq)

@pabigot

Better to have a multi-operation description that steps through a sequence of operations, one of which could be a generic "delay"

I will look into that.

@tbursztyka

And I did not realize this meant that much change in user code to switch to async,

Note that legacy API will keep work and hts221 change was done only to showcase new api. I expect that asynchronous API will go in without any changes in sensors. It will serve for those who wants to use i2c asynchronously (custom driver). It's hard to tell if other vendors switch to that api. Since legacy and async can be mixed, sensor can have most parts common and for example two versions of read out call. However, this is too far into the future as that would require sensor API to be asynchronous.

@pabigot the point of this PR is to evaluate queued operation and sequence manager and show how i2c (spi) can benefit from them. It is also an attempt to fill the main gap I see in spi and i2c - lacking of isr callable functions. Is there a gap when it comes to threads/user space? Is there any issue for that, anyone requesting?

IMO because we're seeing three things at once here it's too hard to assess the I2C parts of this PR

I want to give short presentation on API meeting with overview of this approach.

Notes from 2020-06-30 API telecon.  Edit/reply if I missed something.
TBD: Where the original slides can be obtained.  What's below are my notes.
Presentation from @nordic-krch
Driver: be able to initiate I2C operations from interrupt context.
Requests to make more generic.  Presented overview:

Queued operation manager, allows submission while device is busy (#24781)
Sequence manager with generic actions, notifies when sequence complete (#23075)
Queued sequence, shim to support sequence manager with operation manager (this PR)
I2C API changes using these capabilities (this PR)

Sequence Manager:

Sequence is immutable (pointer to array of actions)
Operation mode:

manager provides processing function
action provides processing function

Shared actions:

pause (invoke callback to allow sequence to resume)
delay (time-based pause)

Macros to:

create the sequence SYS_SEQ_DEFINE
create custom action SYS_SEQ_CUSTOM_ACTION

Queued sequence

queued operation triggers sequence execution

I2C

i2c_single_transfer
i2c_get_async(dev) gets access to the queued operation manager
i2c_async common implementation for actions

Actions:

address set
tx
txv (buffer plus length)
tx_rx
txv_rx

Example using HTS221:

set address, read who-am-i,
pause verify result

Future:

WAIT_FOR_PIN action
Implement for SPI
Combine I2C and SPI (if this makes sense)

JW: likes the script
Questions
Agreement that interrupt-initiation of operations is important?
JF: doesn't seem necessary
KC: low power devices can make progress directly from interrupt context.  Also reduces RAM requirements for dedicated thread.
JF: most now use work queue.  Could use sensor work queue to decouple from impact on system work queue.
Complexity of sequence manager?
PB: The action sequence is getting close to describing a simple finite state machine.  Seems slippery slope: currently linear, looping within actions.  Will this add branching, control structures, ....?  Would that result be maintainable?
KC: Some branching support using return value of action to adjust selection of next action to execute.
Summary
Awaiting further input on the specific questions above and the general concept.  Please comment here or in the specific component PRs.

Presentation: https://docs.google.com/presentation/d/1C6C5Dka_jGIEnTqLROFAs1HTCkK3z_mL64ZH4TzLA-c/edit?usp=sharing

API 2020-07-07: Moved from triage to in-progress pending further activity per https://github.com/zephyrproject-rtos/zephyr/pull/23371#issuecomment-651920879


IE9 (yes I know!) Console Issues

As much as I would like to think that IE9 is dead, it's simply not dead in the corporate world.  Therefore, while we love this plugin, it has to be modified to work with IE9.  Three simple changes have done that - removal of all three "console.log" calls.  IE9 will cause an 'undefined' error and simply not function unless these are hidden or removed.  Have been able to duplicate this over and over again.  Once removed, the issues vanish.
Thought you should know.

Thank you, i will get this done

Thanks..  keep up the great work!


Failed to load the data-explorer package

[Enter steps to reproduce:]

...
...

Atom: 1.32.0 x64
Electron: 2.0.9
OS: Ubuntu 16.04.5
Thrown From: data-explorer package 0.3.1
Stack Trace
Failed to load the data-explorer package
At Cannot find module '@blueprintjs/select'

Error: Cannot find module '@blueprintjs/select'
    at Module._resolveFilename (module.js:543:15)
    at Module._resolveFilename (/usr/share/atom/resources/electron.asar/common/reset-search-paths.js:35:12)
    at Function.get_Module._resolveFilename (/usr/share/atom/resources/app/static/<embedded>:11:154364)
    at Module.require (/app.asar/static/index.js:40:43)
    at require (/usr/share/atom/resources/app/static/<embedded>:11:145671)
    at /packages/data-explorer/node_modules/@nteract/transform-dataresource/lib/VizControls.js:20:15)
    at /packages/data-explorer/node_modules/@nteract/transform-dataresource/lib/VizControls.js:391:3)
    at Module.get_Module._compile (/usr/share/atom/resources/app/static/<embedded>:11:146381)
    at Object.value [as .js] (/usr/share/atom/resources/app/static/<embedded>:11:149928)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)
    at Module.require (/app.asar/static/index.js:47:45)
    at require (/usr/share/atom/resources/app/static/<embedded>:11:145671)
    at /packages/data-explorer/node_modules/@nteract/transform-dataresource/lib/index.js:48:43)
    at /packages/data-explorer/node_modules/@nteract/transform-dataresource/lib/index.js:495:3)
    at Module.get_Module._compile (/usr/share/atom/resources/app/static/<embedded>:11:146381)
    at Object.value [as .js] (/usr/share/atom/resources/app/static/<embedded>:11:149928)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)
    at Module.require (/app.asar/static/index.js:47:45)
    at require (/usr/share/atom/resources/app/static/<embedded>:11:145671)
    at /packages/data-explorer/lib/common.js:33:2)
    at /packages/data-explorer/lib/common.js:11:76)
    at Module.get_Module._compile (/usr/share/atom/resources/app/static/<embedded>:11:146381)
    at Object.value [as .js] (/usr/share/atom/resources/app/static/<embedded>:11:149928)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)

Commands

Non-Core Packages
data-explorer 0.3.1
Hydrogen 2.6.0
hydrogen-launcher 1.2.2
hydrogen-python 0.0.7

I have the same problem when I try to install data-explorer from atom.
Atom 1.32.0 x64
Windows 10 x64
Non-Core Packages
data-explorer 0.3.1
Hydrogen 2.6.0
Hydrogen-python 0.0.7
ide-python 1.0.0
atom-ide-ui 0.13
autocomplete-python 1.10.12
comment 0.6.1

Error: Cannot find module '@blueprintjs/select'

Looks like an issue with the latest release of @nteract/transform-dataresource.
I'll look at this tonight. cc/ @rgbkrk

Ah, looks like @blueprintjs/select needs to be added as a dep on the data explorer.
cc @emeeks

same here

Should be fixed with #3566 unless it's something deeper than that.

Should be fixed with #3566 unless it's something deeper than that.
how can I get the fix? can't find any new commits or a link to the #3566

I'll get the packages shipped today.

@rgbkrk
I installed today the apm install hydrogen data-explorer a atom, and still getting the same error :(

We just shipped a new release of the data explorer that includes it, it's an alpha though. I'll make a PR.

Still getting this same error after installing latest version of data-explorer.


iconfont_convert version too low?

➜ iconfont_convert -c iconfont.yaml
iconfont_convert 1.0.0 requires the Flutter SDK, which is unsupported for global executables.

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.13.3, on macOS 13.5.2 22G91 darwin-arm64, locale zh-Hans-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.2)
[✓] Android Studio (version 2022.2)
[✓] VS Code (version 1.81.1)
[✓] Connected device (3 available)
[✓] Network resources
Do you support this version?

I reproduced the issue you mentioned.
Did you use version A of Flutter to run flutter pub global activate iconfont_convert within the backticks, and then switch to version B of Flutter to run iconfont_convert -c iconfont.yaml?
I reproduced the issue you encountered in this way. Try RERUN flutter pub global activate iconfont_convert to see if it resolves the problem.


Fixed the indentation logic (updated)

Previously the logic lacked handling of multiple matches of the
increase/decrease regexs, so things like the following javascript
would not be handled correctly (the last line would be indented).
if (test) {
  foo({a:1
  })};
var t = "this should be at indentation 0";

Fixed that.
Also revisited the logic regarding decreaseIndent and
decreaseNextIndent. It did not seem very logical that decreaseIndent
did not correspond to increaseIndent: decreaseIndent regarded the
current line whereas increaseIndent regarded the next line (just like
decreaseNextIndent). For that reason I renamed decreaseIndent to
decreaseThisIndent and decreaseNextIndent to decreaseIndent. This way
we also avoid a whole lot of issues related to the concept of
decreaseNextIndent, because most of the language packages I looked at
have not yet adopted it but the semantics of decreaseIndent had
changed. So now the semantics of decreaseIndent is back to what it was
and decreaseThisIndent is the new concept.
Also now using the tokenized line to determine whether a in-/decrease
pattern appears within a quoted string (and ignoring it in that case),
rather than requiring regexes for checking whether or not the
occurrence is within a quoted string or not -- the examples I saw for
that were wrong, too (e.g., things like /} [^"] / are non-sense,
because things like } var x = "hey" are perfectly fine). One really
needs to reason about balanced quotation to get it right, and the
tokenizer already does that for us, so let's use it rather than
reinventing the wheel. The decrease/increase regexes can now be much
simpler in the language packages.
In this version we use decreaseIndentPattern as the default value for decreaseThisIndentPattern, which is OK, because that pattern is never applied (matched) more than once, which was the previous behavior.

Looks like some tests also need to be updated for this.

I don't really think so. This is simply a cyclic dependency with the language packages being used in the tests (in particular the javascript one). In which order do you suggest trying to resolve them? The language-package maintainers may point out the same thing (failing test cases on my PR), which are in turn due to the not-yet-merged PR in atom/atom.

Also created a corresponding PR for language-c:
https://github.com/atom/language-c/pull/110

Thanks for digging into this! It's a pretty tricky area of the code.

For that reason I renamed decreaseIndent to decreaseThisIndent and decreaseNextIndent to  decreaseIndent.

I think we're going to need to keep the existing names, in order to avoid breaking other people's language packages.

I don't really think so.

The tests do need to be updated. We would at least will need to add one or more specs that test the new capabilities that you've added (e.g. dedenting multiple times if multiple } tokens appear on the same line).

the semantics of decreaseIndent had changed

I'm not sure what you mean by this; how have the semantics of decreaseIndent changed? I don't think that they were changed at all when the decreaseNextIndent regexp. was introduced.

Hi Max,
Thanks for attending to this. I saw the dialog you had with @svanharmelen regarding his contributions last summer, so I'm glad we are all on the same page regarding status-quo.
Responding to your points:

renaming: The renaming is incremental. Only two language packages currently set the decreaseNextIndentPattern parameter, go and javascript:

[cfritz@astatine:~/github/atom/node_modules Wed Jan 13 13:39:35]
> grep decreaseNext language-*/settings/*.cson
language-go/settings/language-go.cson:    'decreaseNextIndentPattern': '^\\s*[^\\s()}]+(?<m>[^()]*\\((?:\\g<m>|[^()]*)\\)[^()]*)*[^()]*\\)[,]?$'

(javascript doesn't show up because I've already patched it).
Regarding the renaming of decreaseIndent and decreaseThisIndent: the new code uses the former as the value for the latter if the latter is not defined. This matches the understanding of the language-pack authors at the time they wrote their packages.

Regarding test: again, there are cyclic dependencies with the language packs (in particular javascript, c, and go), which are used in the tests. If the PRs of those got merged then the atom/atom tests would pass. So I'll look to you guys to tell me how to best break this cycle (start with the language packs, atom/atom, or somehow changes the tests temporarily).

changed semantics of decreaseIndent: Yeah, I think you might be right. After looking at more language packs it seems that the regexes were indeed meant for outdenting the current line, which just isn't such a great approach because things like

if (test) {
  var x = 1; }
var shouldBeAtZero;

would become:
if (test) {
var x = 1; }
var shouldBeAtZero;

or (worse), be completely wrong, like:
if (test) {
  var x = 1; }
  var shouldBeAtZero;

The latter is what I have seen in several languages, and which is why my first interpretation of decreaseIndent was actually what decreaseNextIndent is meant to do. Perhaps some of the language pack authors had that same misconception of its semantics.
BTW, please let me know where to document the intended semantics of these parameters. No matter what we decide, we should capture that somewhere so language pack authors can get clarity.

Can you explain a little further about the cyclic dependency? I would think that since your rename was done in a backwards-compatible way, the existing language packages would continue to work. I know that you removed the need for the regexps to check for quotation marks, but I would think that could be done in a backwards compatible way as well (it doesn't hurt to check for quotation marks, it just is no longer necessary).

No they won't because the semantics for increaseIndent has changed (you can now match more than once), and decreaseIndent can also be applied more than once per line. But honestly I don't think the indentation of the language packs is working right now anyways. I think it's quite broken and the tests are just too narrow to show that. I think the number of open issues relating to this show that quite clearly. The regexes for outdenting, e.g., are very brittle. For instance, most of them assume that there is only whitespace before the string/char that indicates the end of a block. For instance in C++ right now you get:
if (x) {
  cout << ok; }

  int thisIsWronglyIndented;

This is because decreaseIndent is really meant for the case where the current line can and has to be outdented already (as with the } in KR style). Since that is not the case above, and since there is no decreaseNextIndent defined for C, the behavior is wrong.
I think we really need to allow ourselves to take a larger scoop here to out get out of this hole. Frankly, inductive (line-to-line) + regex based indentation logic seems error prone to me. I would think that using the parse tree is a better solution approach, so maybe that is something to reconsider, too. But that would be an even longer discussion and larger change.

The only way I see to make this merge not require simultaneous changes in the language packs is if we literally set some sort of flag in the language pack settings indicating to use the new semantics, and then keep the old logic around and use it for any language packs that do not set that flag to true. I think it's a little ugly, but perhaps as a transitional and temporary step that's OK.
Would you prefer that?

@maxbrunsfeld could you let me know how you want us to move forward with this? I don't want this to be forgotten and many people are waiting on a fix to the indentation problems in several languages.
Please let me know which path forward you think is the most promising, both in terms of long-term correctness of the logic and short-term merge-ability:

merge the current PRs (this one and the language ones) and create corresponding PRs for all other languages. Pro: getting to the right solution faster. Con: intermediate even more broken indentation logic and failing tests.

change the logic to detect somehow (perhaps using a flag in the settings) whether to use the old or new indentation logic. Pro: no disruptions whatsoever. Con: some ugly code for some time.

write a completely new indentation logic, e.g., based on tokens (I'm actually toying with this right now using emacs as the baseline). Con: same as 2. Pro: potentially much better indentation (emacs indentation is pretty fantastic), including for multi-line expressions such as:

var x = someVeryLongVariableNameThatIsLooooooooooooooooooooooooooooooooooooong
  *<PHONE_NUMBER>36457165328762587365487623548756237847523948572345
  / numberOfElementsInTheUniverse;

// and pipelines:
svg.selectAll("g")
    .data(tiler
      .scale(projection.scale() * 2 * Math.PI)
      .translate(projection([0, 0])))
  .enter().append("g")
    .each(function(d) {
      var g = d3.select(this);
      d3.json("http://" + ["a", "b", "c"][(d[0] * 31 + d[1]) % 3] + ".tile.openstreetmap.us/vectiles-highroad/" + d[2] + "/" + d[0] + "/" + d[1] + ".json", function(error, json) {
        g.selectAll("path")
           .data(json.features.sort(function(a, b) { return a.properties.sort_key - b.properties.sort_key; }))
           .enter().append("path")
           .attr("class", function(d) { return d.properties.kind; })
           .attr("d", path);
      });
    });

The latter would currently be indented as:
var x = someVeryLongVariableNameThatIsLooooooooooooooooooooooooooooooooooooong
*<PHONE_NUMBER>36457165328762587365487623548756237847523948572345
/ numberOfElementsInTheUniverse;

// and pipelines:
svg.selectAll("g")
.data(tiler
   .scale(projection.scale() * 2 * Math.PI)
   .translate(projection([0, 0])))
.enter().append("g")
.each(function(d) {
   var g = d3.select(this);
   d3.json("http://" + ["a", "b", "c"][(d[0] * 31 + d[1]) % 3] + ".tile.openstreetmap.us/vectiles-highroad/" + d[2] + "/" + d[0] + "/" + d[1] + ".json", function(error, json) {
     g.selectAll("path")
     .data(json.features.sort(function(a, b) { return a.properties.sort_key - b.properties.sort_key; }))
     .enter().append("path")
     .attr("class", function(d) { return d.properties.kind; })
     .attr("d", path);
  });
});

Please let me know your preference so I know whether it's worth the trouble implementing this further.

Please let me know which path forward you think is the most promising:

This is what I'd like to do:

Revert the naming changes that you've introduced to the settings. In Atom, we try not to introduce unnecessary API changes, in order to provide a consistent platform for packages. In some cases, changes are necessary, but I don't see a strong reason for to change the names in this case.

See what specs fail after the original names are restored. I'm still not understanding why these changes (without the renames) can't be done in a backwards-compatible way. What would help me to understand is if you could take a look at the test failures on Travis, and explain specifically why one or more individual tests are failing.

If there are indeed unavoidable spec failures, open PRs on the relevant language packages with the minimal changes required to make the tests pass (no setting renames, just regexp changes or setting additions). I will publish pre-release versions of those packages (e.g. v0.54.0-test-indentation-fixes) based on those PRs' branches.

On this branch of atom/atom, as part of this PR, update Atom's package.json to point to those pre-release versions.

See the tests go green.

write a completely new indentation logic, e.g., based on tokens

I agree with you that this would be a better approach.
I'm a little bit concerned that Atom's current tokenization is not reliable enough to provide truly correct indentation either. In the next year, we plan to start rolling out a new parsing system in Atom, one language at a time. The new system will provide an exact syntax tree for the program, instead of a sequence of tokenized lines based on regex matches. As we roll out that system, I am very excited about updating Atom's auto indentation system to make use of the syntax tree. I'm concerned that until that time, a complete rewrite of Atom's indentation system which will require changes to all bundled and third-party language packages may not be worth it.
That said, I don't want to discourage you from experimenting! Just be aware that such a PR will probably be very difficult to land.

Awesome, @maxbrunsfeld, I'll proceed as suggested.
Great to hear about the upcoming syntax trees. That did come to my mind very quickly as well as I was thinking more about it. I agree that that will lend itself to a much better logic. In that case I'll put my token-based work on hold and just prepare the line up of the PRs as suggested for now.
BTW, the short answer to 2. is that in the new logic the same regex can be matched multiple times per line, so
foo({ // one one opening scope is recognized
  a: 1,
  b: 2
  },
c);  // this is wrong

var d; // this would be at indentation -1, which is prevented by Math.max(desiredIndentLevel, 0)

now becomes
foo({
    a: 1,
    b: 2
  },
  c);

Which is correct, and no need to rely on max to save us.
However, the test cases may desire the first behavior. But I'll investigate further. I should be able to make the tests go green.

@maxbrunsfeld When you say In the next year, we plan to start rolling out a new parsing system in Atom do you mean next year (2017) or this (2016) year?

I meant 2016, but as always, that is a very rough estimate.

@maxbrunsfeld I've renamed the regexes back and the tests seem to pass again. But now there seems to be some issue with the testing procedure itself, or something like that. If you look at the log of why the checks fail you'll see:
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

Any idea how to fix that? I can't see how this is related to my code.

Travis can be flaky at times...I've restarted the build.

Thanks, @50Wliu. Travis got past that issue now (it only seems to happen when checking the specs takes more than 10minutes). Now there is another error that I don't know how to interpret.

Now there is another error that I don't know how to interpret.

Another case of flakiness that you can safely ignore :)

@chfritz Thanks for keeping at this. It's looking like a very tightly-focused change now :+1:. Can you add one or more tests that demonstrate the new and improved indentation? The example that you gave in your initial PR description seems like it would make a good test. I think that there should be a test that each regexp (increaseIndent, decreaseIndent, and decreaseNextIndent) can match multiple times.

@maxbrunsfeld : with pleasure! I was thinking the same.

ok, @maxbrunsfeld, I added three cases, one each for checking a case where one of the three regexs matches multiple times.

Wow...thanks for smoothing this out. Fantastic job.
Regarding the last discussed example:
fs.readFile('/etc/passwd', (err, data) => {
    // indented two levels
});

Personally, I would much rather have the two-level indentation and overall "more correct" indentation (i.e., indentation I have to spend less time correcting). If the code with the function is all indented correctly, I certainly would not mind the extra indentation for the entire block.

I'm going to close this one out because we never really reached consensus on a way forward.
By the way, syntax trees have been available in Atom for a while now, if you enable the core.useTreeSitterParsers setting. We are pretty close to enabling this setting by default. In the next few months, I'd like to start designing a new auto-indent system that takes advantage of these syntax trees. I'll try to remember to CC some of you on any issues that I open.

@maxbrunsfeld I've taken a look at what the new tree-sitter parser produces. Pretty sweet! Since my sane-indentation package was broken anyways, due to some recent changes I hadn't yet caught up with, I decided to "fast-forward" to tree-sitter and rebuild the package based on that, at least for JavaScript. Thought you might be curious to give it a try. Note that it sets core.useTreeSitterParsers to true.
I think it's working pretty well. My biggest issue right now is dealing with parse-errors. When the program doesn't parse, then the entire indentation logic is broken, which only makes it harder to fix the syntax error in one's program. This is a bummer, because I always felt that good indentation is the key to finding syntax errors quickly. Any thoughts on how to overcome that problem are welcome!


Crazy4pi314/katas improvements

Taking a stab at improving the BB84 kata by adding some task descriptions as well as adding more introduction. I also removed references to polarization to keep the description of the protocol implementation agnostic.
There are also a couple of open issues for this kata (https://github.com/microsoft/QuantumKatas/issues/296 and https://github.com/microsoft/QuantumKatas/issues/295) that I would happy to help fix if that is helpful, but I wanted to check if that was ok first.

Looks like there is some error in the notebook structure itself - CI build fails with "Notebook does not appear to be JSON" error, and I can't open it locally. Could you please take a look?

Looks like there is some error in the notebook structure itself - CI build fails with "Notebook does not appear to be JSON" error, and I can't open it locally. Could you please take a look?

I got that fixed sorry. Looks like Github suggestions garbled a bit of syntax from the suggestions.

Meh, just realized that my fix to the reference implementation was incorrect, changing the condition of the bits we're counting is sufficient. That's what I get for working after 11 pm :-) Let me add a test to catch this error definitively.


⚠ Jenkins Plugins Artifactory has degraded performance

In 7980772, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1179 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 401e883 after 12 minutes.


[SVG] Better handling of empty  element

Before, when compiling from XML, only <colorPalettes></colorPalettes> would be accepted.
Now, all of these pass compilation as well,
<colorPalettes>    </colorPalettes>

<colorPalettes />

<colorPalettes>
</colorPalettes>

What if it's absent? See https://github.com/behdad/fonttools/issues/531

@RoelN thanks for pointing that out. I'll see what I can do.

OK, it now compiles and decompiles without the <colorPalettes> element.

Coverage increased (+0.004%) to 70.693% when pulling b9fc0b84ed1a02231271b3af2943e7375df24af3 on miguelsousa:svg-colorPalettes-fix into 967cf8bb15e015bc0a75dc1f89ad5e5bbc5f1571 on behdad:master.


Fehlermeldung bei go-e Charger

Describe the bug

Ich bekomme wieder eine Fehlermeldung, die mit einem älteren Repo-Stand eigentlich weg war. --> ERROR 2020/07/16 20:36:21 charge controller error: amp update failed
Der SOC wird nicht mehr angezeigt und auch keine Ladeleistung
Egal welchen Knopf ich drücke (Sofort laden / Min+PV), es wird immer mit min current = 9A geladen (Infos am Smartmeter Portal)

To Reproduce
evcc mit Stand vom 16.07.2020 18:00 Uhr
kompilliert und in Betrieb auf Ubuntu Server
Expected behavior
Wahl der Ladesteuerung sollte wieder gehen und
Anzeige des SOC und der Ladeleistung sollte wieder gehen
--> Hinweis: SOC mittels APP wird angezeigt und "./evcc vehicle" liefert auch den SOC
EVCC details:
Show output of evcc -v
evcc version 0f02f6f (0f02f6f)
Show evcc log output with --log trace
Jul 16 20:36:21 oeko-t evcc[19408]: [cache] TRACE 2020/07/16 20:36:21 chargedEnergy: 0  Jul 16 20:36:21 oeko-t evcc[19408]: [cache] TRACE 2020/07/16 20:36:21 chargeDuration: 0s  Jul 16 20:36:21 oeko-t evcc[19408]: [cache] TRACE 2020/07/16 20:36:21 socCharge: -1  Jul 16 20:36:21 oeko-t evcc[19408]: [cache] TRACE 2020/07/16 20:36:21 chargeEstimate: -1  Jul 16 20:36:21 oeko-t evcc[19408]: [iflx] TRACE 2020/07/16 20:36:21 write chargedEnergy=0 (map[loadpoint:go-e Charger])  Jul 16 20:36:21 oeko-t evcc[19408]: [go-e] TRACE 2020/07/16 20:36:21 https://api.go-e.co/api_status?token=xxxxxxxxxxxx  Jul 16 20:36:21 oeko-t evcc[19408]: {"success":true,"age":3730,"data":{"version":"B","tme":"1607202036","rbc":"26","rbt":"341442642","car":"1","amp":"9","err":"0","ast":"0","alw":"1","stp":"0","cbl":"32","pha":"56","tmp":"22","tma":[22,21.5,21.75,21.37],"amt":"32","dws":"0","dwo":"0","adi":"0","uby":"0","eto":"4170","wst":"3","txi":"1","nrg":[225,224,227,1,0,0,0,0,0,0,0,0,0,0,0,0],"fwv":"033","sse":"010486","wss":"G-Hof","wke":"","wen":"1","cdi":"0","tof":"101","tds":"1","lbr":"148","aho":"3","afi":"7","azo":"0","ama":"32","al1":"10","al2":"16","al3":"20","al4":"24","al5":"32","cid":"255","cch":"16748800","cfi":"65280","lse":"1","ust":"2","wak":"","r1x":"2","dto":"0","nmo":"0","sch":"AAAAAAAAAAAAAAAA","sdp":"0","eca":"0","ecr":"40","ecd":"0","ec4":"0","ec5":"0","ec6":"0","ec7":"0","ec8":"0","ec9":"0","ec1":"0","rca":"6414","rcr":"1719","rcd":"","rc4":"","rc5":"","rc6":"","rc7":"","rc8":"","rc9":"","rc1":"","rna":"G\\u00e4ste","rnm":"Tjarko","rne":"ZOE","rn4":"","rn5":"","rn6":"","rn7":"","rn8":"","rn9":"","rn1":"","loe":0,"lot":0,"lom":0,"lop":0,"log":"","lon":0,"lof":0,"loa":0,"lch":28650,"upd":"0"}}  Jul 16 20:36:21 oeko-t evcc[19408]: [lp-1] DEBUG 2020/07/16 20:36:21 charger status: A  Jul 16 20:36:21 oeko-t evcc[19408]: [lp-1] DEBUG 2020/07/16 20:36:21 set charge current: 9A  Jul 16 20:36:21 oeko-t evcc[19408]: [cache] TRACE 2020/07/16 20:36:21 connected: false  Jul 16 20:36:21 oeko-t evcc[19408]: [cache] TRACE 2020/07/16 20:36:21 charging: false  Jul 16 20:36:21 oeko-t evcc[19408]: [go-e] TRACE 2020/07/16 20:36:21 https://api.go-e.co/api?token=xxxxxxxxxxx&payload=amp=9 Jul 16 20:36:21 oeko-t evcc[19408]: {"success":true,"payload":"amp=9"}  Jul 16 20:36:21 oeko-t evcc[19408]: [lp-1] ERROR 2020/07/16 20:36:21 charge controller error: amp update failed: 0  Jul 16 20:36:21 oeko-t evcc[19408]: [core] DEBUG 2020/07/16 20:36:21 lp-1 remaining power: 691W = 691W - 0W J ul 16 20:36:21 oeko-t evcc[19408]: [cache] TRACE 2020/07/16 20:36:21 error: [lp-1] ERROR 2020/07/16 20:36:21 charge controller error: amp update failed: 0\n

Mist, erst 2min nach deinem ticket...

Sorry, da hatte sich unsere Kommunikation gerade überschnitten.
Fazit: habe den fix/go-e branch kompilliert und getestet. Nun

sudo tail /var/log/syslog
Jul 16 22:11:57 oeko-t evcc[680]: [go-e] TRACE 2020/07/16 22:11:57 https://api.go-e.co/api_status?token=xxxxxxxx
Jul 16 22:11:57 oeko-t evcc[680]: {"success":true,"age":1790,"data":{"version":"B","tme":"1607202211","rbc":"26","rbt":"347180045","car":"1","amp":"9","err":"0","ast":"0","alw":"1","stp":"0","cbl":"32","pha":"56","tmp":"22","tma":[22.37,21.75,22.37,22.25],"amt":"32","dws":"0","dwo":"0","adi":"0","uby":"0","eto":"4190","wst":"3","txi":"1","nrg":[226,226,227,1,0,0,0,0,0,0,0,0,0,0,0,0],"fwv":"033","sse":"010486","wss":"G-Hof","wke":"","wen":"1","cdi":"0","tof":"101","tds":"1","lbr":"148","aho":"3","afi":"7","azo":"0","ama":"32","al1":"10","al2":"16","al3":"20","al4":"24","al5":"32","cid":"255","cch":"16748800","cfi":"65280","lse":"1","ust":"2","wak":"","r1x":"2","dto":"0","nmo":"0","sch":"AAAAAAAAAAAAAAAA","sdp":"0","eca":"0","ecr":"40","ecd":"0","ec4":"0","ec5":"0","ec6":"0","ec7":"0","ec8":"0","ec9":"0","ec1":"0","rca":"6414","rcr":"1719","rcd":"","rc4":"","rc5":"","rc6":"","rc7":"","rc8":"","rc9":"","rc1":"","rna":"G\u00e4ste","rnm":"Tjarko","rne":"ZOE","rn4":"","rn5":"","rn6":"","rn7":"","rn8":"","rn9":"","rn1":"","loe":0,"lot":0,"lom":0,"lop":0,"log":"","lon":0,"lof":0,"loa":0,"lch":1485,"upd":"0"}}
Jul 16 22:11:57 oeko-t evcc[680]: [lp-1] ERROR 2020/07/16 22:11:57 charge controller error: json: invalid use of ,string struct tag, trying to unmarshal "033" into string
Jul 16 22:11:57 oeko-t evcc[680]: [core] DEBUG 2020/07/16 22:11:57 lp-1 remaining power: 1982W = 1982W - 0W
Jul 16 22:11:57 oeko-t evcc[680]: [cache] TRACE 2020/07/16 22:11:57 error: [lp-1] ERROR 2020/07/16 22:11:57 charge controller error: json: invalid use of ,string struct tag, trying to unmarshal "033" into string\n
`

behoben!

Danke! SOC ist wieder da und Steuerung funtioniert auch. Aber es wurde keine Push-Nachricht beim beenden gesendet, was sicherlich mit folgender Meldung zusammenhängt:

Liegt der Fehler hier noch auf meiner Seite in der yaml?

 Z.b. Energy -> charegedEnergy. Siehe dist.yaml.


Tensorflow Lite does not support tf.math.rsqrt() operation conversion.

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 64bit
TensorFlow installed from (source or binary): Python 3.8.2, pip: 22.1, pip package
TensorFlow version (or github SHA if from source): TF version:'2.9.0'

Provide the text output from tflite_convert
# Copy and paste here
2022-09-29 14:49:40.084984: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-09-29 14:49:40.575045: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 2807 MB memory:  -> device: 0, name: NVIDIA GeForce MX230, pci bus id: 0000:01:00.0, compute capability: 6.1
WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. `model.compile_metrics` will be empty until you train or evaluate the model.
C:\Python38\lib\site-packages\tensorflow\lite\python\convert.py:766: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway.
  warnings.warn("Statistics for quantized inputs were expected, but not "
2022-09-29 14:49:40.988706: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-09-29 14:49:40.989008: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2022-09-29 14:49:40.990306: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: C:\Users\DELL\AppData\Local\Temp\tmpv6cmtx6m
2022-09-29 14:49:40.993266: I tensorflow/cc/saved_model/reader.cc:81] Reading meta graph with tags { serve }
2022-09-29 14:49:40.993545: I tensorflow/cc/saved_model/reader.cc:122] Reading SavedModel debug info (if present) from: C:\Users\DELL\AppData\Local\Temp\tmpv6cmtx6m
2022-09-29 14:49:40.997743: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2022-09-29 14:49:40.998221: I tensorflow/cc/saved_model/loader.cc:228] Restoring SavedModel bundle.
2022-09-29 14:49:41.024607: I tensorflow/cc/saved_model/loader.cc:212] Running initialization op on SavedModel bundle at path: C:\Users\DELL\AppData\Local\Temp\tmpv6cmtx6m
2022-09-29 14:49:41.030886: I tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 40573 microseconds.
2022-09-29 14:49:41.037367: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:263] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2022-09-29 14:49:41.053320: I tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1972] Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs

Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs
fully_quantize: 0, inference_type: 6, input_inference_type: 9, output_inference_type: 9
error: illegal scale: INF

Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
code:
import tensorflow as tf
import numpy as np
import pathlib

def make_model():
    data = tf.constant(np.arange(60000).reshape(200,10,10,3) /60000, dtype=tf.float32)
    input_var=tf.keras.Input(shape=[10,10,3])
    out_var=tf.math.rsqrt(input_var)
    model=tf.keras.models.Model(input_var,out_var)
    model(data)
    return model,data

def save_model(model,path="output/sqrt_H5/"):
    tf.keras.models.save_model(model,path+"H5/SQRT.h5")

def convert_model(model,data):
    def representative_data_gen():
            for input_value in data:
                input_value = input_value[np.newaxis, ...]
                yield [input_value] # shape should be (1, <data point size))
    converter = tf.lite.TFLiteConverter.from_keras_model(model)
    converter.optimizations = [tf.lite.Optimize.DEFAULT]
    converter.representative_dataset = representative_data_gen
    # Ensure that if any ops can't be quantized, the converter throws an error
    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]

    converter.inference_input_type = tf.int8
    converter.inference_output_type = tf.int8

    tflite_model = converter.convert()
    tflite_models_dir = pathlib.Path("output/tflite_models/")
    tflite_models_dir.mkdir(exist_ok=True, parents=True)
    tflite_model_file = tflite_models_dir/"rsqrt_int.tflite"
    tflite_model_file.write_bytes(tflite_model)

def Execute():
    holder,data=make_model()
    save_model(holder)
    convert_model(holder,data)

Also, please include a link to a GraphDef or the model if possible.
Any other info / logs
tried to create  a single rsqrt operation containing model saved its .h5 with no problem and then convert it to its int8 .tflite but got the above error.
netron of hdf5 save is as follows

Include any logs or source code that would be helpful to diagnose the problem.
If including tracebacks, please include the full traceback. Large logs and files
should be attached.

Hi @Saar-Ken-Ji !
In Colab , Session was crashing in Colab with 2.9 .
rsqrt in select ops list. So could you try with below syntax in lite conversion.
    converter.target_spec.supported_ops = [
     tf.lite.OpsSet.TFLITE_BUILTINS_INT8, # enable TensorFlow Lite ops.
     tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
    ]

    converter.inference_input_type = tf.int8
    converter.inference_output_type = tf.int8
    converter.experimental_enable_quantizer = True

Hi @mohantym,
Thank you for the quick response and your help on this.
Does this mean that the tflite default int8 built-ins don't support rsqrt and we need to use SELECT_TF_OPS for converting a model which has rsqrt ? Please confirm.

Hi  @mohantym
thank you for the timely response . i have done as you have advised. but the error remains as follows.
2022-09-29 16:40:16.914627: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-09-29 16:40:17.407041: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 2807 MB memory:  -> device: 0, name: NVIDIA GeForce MX230, pci bus id: 0000:01:00.0, compute capability: 6.1
WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. `model.compile_metrics` will be empty until you train or evaluate the model.
C:\Python38\lib\site-packages\tensorflow\lite\python\convert.py:766: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway.
  warnings.warn("Statistics for quantized inputs were expected, but not "
2022-09-29 16:40:17.833677: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-09-29 16:40:17.833808: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2022-09-29 16:40:17.835122: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: C:\Users\DELL\AppData\Local\Temp\tmplgj34q1q
2022-09-29 16:40:17.837976: I tensorflow/cc/saved_model/reader.cc:81] Reading meta graph with tags { serve }
2022-09-29 16:40:17.838086: I tensorflow/cc/saved_model/reader.cc:122] Reading SavedModel debug info (if present) from: C:\Users\DELL\AppData\Local\Temp\tmplgj34q1q
2022-09-29 16:40:17.840627: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2022-09-29 16:40:17.841149: I tensorflow/cc/saved_model/loader.cc:228] Restoring SavedModel bundle.
2022-09-29 16:40:17.857504: I tensorflow/cc/saved_model/loader.cc:212] Running initialization op on SavedModel bundle at path: C:\Users\DELL\AppData\Local\Temp\tmplgj34q1q
2022-09-29 16:40:17.863103: I tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 27977 microseconds.
2022-09-29 16:40:17.869406: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:263] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2022-09-29 16:40:17.884644: I tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1972] Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs

Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs
fully_quantize: 0, inference_type: 6, input_inference_type: 9, output_inference_type: 9
error: illegal scale: INF

@Saar-Ken-Ji !
It seems rsqrt has also been added in built in ops along select ops allow list.
Could you skip the representative dataset and let us know after trying  with inference input and output as float 32 type .
To debug quantization issues (Delist quantization for problematic layers) , You can use quantization_debugger .
Thank you!

hello thank you for your suggestion, i have done as you have advised.
I was able to get the default quantization result, with input and output as float32 as you had suggested. but what i require is the int8 quantization of the same. is it not possible to achieve that?
note: using the quantization_debugger also produced the same error as mentioned before as the debugger also require representative data set even in case of default quantization. so is it an issue of not being able to process that in the module?

@Saar-Ken-Ji !
Could you share your quantization debugger results error here.
Thank you!

@mohantym
the bellow given is the error when i try default quantization with debuggers debug data set  parameter assigned with the representative data set from the code above
2022-09-30 14:10:35.736988: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-09-30 14:10:36.286672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 2807 MB memory:  -> device: 0, name: NVIDIA GeForce MX230, pci bus id: 0000:01:00.0, compute capability: 6.1
WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. model.compile_metrics will be empty until you train or evaluate the model.
C:\Users\DELL\AppData\Roaming\Python\Python38\site-packages\tensorflow\lite\python\convert.py:766: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway.
warnings.warn("Statistics for quantized inputs were expected, but not "
2022-09-30 14:10:36.730492: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-09-30 14:10:36.730963: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2022-09-30 14:10:36.733999: I tensorflow/cc/saved_model/reader.cc:45] Reading SavedModel from: C:\Users\DELL\AppData\Local\Temp\tmp_b1f8opg
2022-09-30 14:10:36.735469: I tensorflow/cc/saved_model/reader.cc:89] Reading meta graph with tags { serve }
2022-09-30 14:10:36.735881: I tensorflow/cc/saved_model/reader.cc:130] Reading SavedModel debug info (if present) from: C:\Users\DELL\AppData\Local\Temp\tmp_b1f8opg
2022-09-30 14:10:36.739485: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2022-09-30 14:10:36.740029: I tensorflow/cc/saved_model/loader.cc:229] Restoring SavedModel bundle.
2022-09-30 14:10:36.755296: I tensorflow/cc/saved_model/loader.cc:213] Running initialization op on SavedModel bundle at path: C:\Users\DELL\AppData\Local\Temp\tmp_b1f8opg
2022-09-30 14:10:36.770024: I tensorflow/cc/saved_model/loader.cc:305] SavedModel load for tags { serve }; Status: success: OK. Took 36022 microseconds.
2022-09-30 14:10:36.784119: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:268] disabling MLIR crash reproducer, set env var MLIR_CRASH_REPRODUCER_DIRECTORY to enable.
2022-09-30 14:10:36.802819: I tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1989] Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs
Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs
fully_quantize: 0, inference_type: 6, input_inference_type: FLOAT32, output_inference_type: FLOAT32
error: illegal scale: INF

Ok @Saar-Ken-Ji !
Thanks for the update.
@sachinprasadhs !
Could you look at this issue.
Thank you!

RSQRT is supported as builtin operator, could you please check the issue at https://github.com/tensorflow/tensorflow/issues/58048.
Also, refer the comment here for the code file reference to the registered RSQRT builtin op.

@sachinprasadhs
thank you for the response.
as per the discussion.
I have gone through the documentation of the rsqrt() and observed that the issue was that the operator couldn't handle negative values  as input so adding a tf.math.abs() before rsqrt solved the issue.
thank you very much for your guidance.


[qtwebengine] Build error on x64-windows

Package<EMAIL_ADDRESS>Host Environment

Host: x64-windows
vcpkg-tool version: 2024-07-10-d2dfc73769081bdd9b782d08d27794780b7a99b9
vcpkg-scripts version: b2cb0da53 2024-11-15 (2 weeks ago)

To Reproduce
vcpkg install --editable qtwebengine
Failure logs
-- Using cached pypa-get-pip-24.2.tar.gz.
-- Extracting source D:/vcpkg/downloads/pypa-get-pip-24.2.tar.gz
-- Using source at D:/b/qtwebengine/src/24.2-36e81c02c7
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:127 (message):
    Command failed: D:/vcpkg/downloads/tools/python/python-3.12.7-x64/python.exe D:/b/qtwebengine/src/24.2-36e81c02c7/public/get-pip.py --no-warn-script-location
    Working Directory: D:/b/qtwebengine
    Error code: Access violation
    See logs for more information:

Call Stack (most recent call first):
  installed/x64-windows/share/vcpkg-get-python-packages/x_vcpkg_get_python_packages.cmake:44 (vcpkg_execute_required_process)
  ports/qtwebengine/portfile.cmake:68 (x_vcpkg_get_python_packages)
  scripts/ports.cmake:196 (include)

So, it seems import pip._internal.commands crashes the python interpreter:
PS D:\> D:/vcpkg/downloads/tools/python/python-3.12.7-x64/python.exe
Python 3.12.7 (tags/v3.12.7:0b05ead, Oct  1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, 'D:\\pip')
>>> import pip
>>> import pip._internal
>>> import pip._internal.commands
PS D:\> Write-Host $?
False
PS D:\>

This probably belongs in the current group of issues with system python vs. vcpkg python3, on Windows.

It is strange that vcpkg provides two kinds of python deployment. One that is compiled and installed as a package via vcpkg install python3 and the other one in tools which is seemingly the embeddable package.
Removing the environment variables pointing to the system python does not solve the issue, however removing the entire installation does.
I would like to have the option to select which python deployment vcpkg uses and possibly keep my existing installation.

You shouldn't need to remove the existing installation. It is an isolation problem, maybe solved by https://github.com/microsoft/vcpkg/pull/41874.
The compiled and installed version, port python3, is for everything which ABI-wise depends on, or leads to dependencies on, python. ABI tracking is based on ports. Selecting another python must be done by overriding that port.

@soroush Is your issue solved now?


When updating a nested object's property to 'undefined', the property is not updated

When updating a nested object's property to 'undefined', the property is not updated.
It only gets updated when providing null as property value, or when omitting the property from the object passed to the assign function.
I'm curious if this is intended behavior or a bug?
I created a failing test demonstrating the behavior here: https://github.com/rbuhrs/mikro-orm/blob/d5eefb2d30e3bb29ee122b7858187a0150541733/tests/issues/GH_new_issue.test.ts

Sounds like a bug in em.assign, does it work when you modify the property directly?

@B4nan thanks for fixing 👍
I do have a follow-up question: I noticed that when a nested property is omitted in the data passed to assign, it is not removed.
https://github.com/rbuhrs/mikro-orm/blob/d57c35db8d335ea9e99292a43dbfb93df32c1c6b/tests/features/entity-assigner/GH4428.test.ts#L136
https://github.com/rbuhrs/mikro-orm/blob/d57c35db8d335ea9e99292a43dbfb93df32c1c6b/tests/features/entity-assigner/GH4428.test.ts#L155
I suspect this is by design ? As this behaviour looks the same as in the entity helper docs when using mergeObjects: true (which is the default), and also is how the regular Object.assign function works.
If this is the intended behaviour, would it then be correct to say that:
a) when using mergeObjects, only the props/values which are passed as data are subject to change (including props provided with undefined)
b) when not using mergeObjects, the entity is changed so that it reflects the object passed as data

Yes

@B4nan Just flagging that this most likely was a breaking change. Queries like the following used to work fine pre 5.7 and would return the required value if condition was truthy:
repo.findOneOrFail(
        {
          id,
          someProp: condition ?  undefined : { some: value }
        },
)

I believe that fixing this issue caused those queries to now compile down to someProp is null. Not trying to hijack this issue, just posting for posterity that this shouldn't have been included in a minor version bump I believe.

@johanneslumpe I believe it always behaved like that, we never ignored undefined keys. I can see the very same behavior in v5.6 as well as v5.5.

@B4nan Sorry for not getting back to this earlier! I will have to try and reproduce this as I was quite certain that this was breaking. If not, my bad. Definitely open to discussion a potential change! I'll try to get back to you with findings as soon as I can.


Fix issue with connection being closed by load balancer aws elb

This fix is configurable by enabling/disabling through environment variable at drone server:

DRONE_CLOSE_NOTIFY_TIMEOUT=60s

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

I think the enable flag should be dropped and it should be enabled with a value >0 and disabled with the value 0.

Not an expert related to that part but LGTM

This is working quite well and all you have to do is at AWS load balancer, set idle timeout same as DRONE_CLOSE_NOTIFY_TIMEOUT and for jobs that takes time you have to increase aws idle timeout and set flag to that value, if job processing time is grater than idle timeout, you don't see the feeds/logs on the UI and you need to restart drone server in order to receive those logs, but the job will be completed at agent side.
The whole problem with idle timeout is that AWS load balancer has a limit of keeping connection open and unfortunately drone server does not send heartbeat/ping-pong to keep this TCP connection live for pulling api other long pulling apis (except websocket apis) and AWS ELB detects that there was not data going through this connection and it will close connection, but the drone server is not aware of that close connection, so we need to handle that case and the fix is on this commit.
Read more about that here http://felixge.de/2014/08/26/tcp-keepalive-with-golang.html
and here http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/config-idle-timeout.html
Thanks.

closing because the master branch no longer uses long polling, and instead uses the drone/mq package with a websocket transport. This should therefore resolve this particular problem. How to configure ELB for websockets is a separate issue though :)


Update python todo apps to use Azure SDK identity preview 1.13.0b2 with AzureDeveloperCredential

Updates python templates to use the AzureDeveloperCredential from the Azure SDK lib

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-python-mongo-aca
Remote: azure-samples-staging
Branch: pr/1574
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-aca -b pr/1574

View Changes | Compare Changes

Project: todo-python-mongo-swa-func
Remote: azure-samples-staging
Branch: pr/1574
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-swa-func -b pr/1574

View Changes | Compare Changes

Project: todo-python-mongo
Remote: azure-samples-staging
Branch: pr/1574
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo -b pr/1574

View Changes | Compare Changes

Project: todo-python-mongo-terraform
Remote: azure-samples-staging
Branch: pr/1574
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-terraform -b pr/1574

View Changes | Compare Changes

CC @pamelafox


complete api

This fixes previous api inconsistencies, and adds compose, applyMiddleware, and bindActionCreators.
I've done limited testing and things seem to work.
Sidenote: it may be worth trying to use the redux tests against minidux.

This also adds testron for testing.

@sethvincent nice! code looks great and confirmed all tests run


Ignore fusion-plugin-csrf-protection-react

This package is obsolete as of fusion-plugin-csrf-protection 2.x

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Output window bottom scroll bar cropping text

Description:
The bottom of the output window is cropped

I propose adding some extra space before the scrolling bar.
Steps to Reproduce:
Print long text outputs inline
Versions:
Atom    : 1.14.1
Electron: 1.3.13
Chrome  : 52.0.2743.82
Node    : 6.5.0

We probably missed this one because on macOS the scrollbars are hidden by default :smile:
I don't know if there is a standard way of reserving space for scrollbars.
Would you like to make a PR to fix this?


Realsense_viewer massive memory leak and crash with Intel graphics

Required Info

Camera Model
D455

Firmware Version
<IP_ADDRESS>

Operating System & Version
Win 10  21H1

Platform
Lenovo Thinkpad X1 Gen 6, Alienware R14, others

SDK Version
2.44 - 2.50

Issue Description
As soon as I start the depth stream in the viewer, RAM memory use starts going up.  When RAM is maxed (16GB in a few minutes) the viewer framerate and overall Windows responsiveness starts dropping as paging memory starts getting used.  This happens much faster with the 3D display than 2D, but it still happens in 2D.  Eventually what usually happens is Windows then blue screens, sometimes viewer just crashes.
I have been able to duplicate this using the standalone viewer going back to 2.39, have not tried earlier as not sure firmware is compatible.  This happens on multiple PCs running different versions of Win 10.
An interesting thing is that it appears to be related to running on the integrated Intel graphics; on the Alienware, when I explicitly set viewer to run on the GTX1060 using nVidia Control Panel, the problem stopped.  It also does not happen on non-laptop PCs where I am directly using nVidia cards.
I run all sorts of graphics apps on these laptops, and this is the only program doing this.  Even programs using the depth stream via the SDK work fine - this is a viewer-only problem as far as I have seen.
My suspicion is that it is coming from the display portion of the viewer, since SDK programs work fine, and switching to 3D from 2D in the viewer accelerates the memory use.

Hi @noirflux  I have seen a memory leak occur in the Viewer before as a consequence of the GLSL feature being enabled (it is enabled by default), as described in https://github.com/IntelRealSense/librealsense/issues/8110#issuecomment-754716585
Could you try disabling the two GLSL options in the Viewer's settings interface using the instructions at https://github.com/IntelRealSense/librealsense/issues/8110#issuecomment-754705023 to see whether this resolves your memory leak, please?

Yes, turning off the GLSL option stops the memory leak.  Leaving just the GLSL rendering option on, still no memory leak.  Turning on just the GLSL processing option may cause a memory leak, but it is very slow in comparison to them both being on.
Interestingly, when I turned off both options after first running with them both on, and 8GB of memory had been consumed, the memory was freed when I clicked OK in that dialogue box.  So this is an obvious repeatable bug.

Problems such as memory leaks or disappearing point cloud when GLSL is enabled in the Viewer is a repeatable issue and there is an existing official Intel bug report about it.  There are no further details that I can provide regarding a timetable for a fix though.

Hi @noirflux  Do you require further assistance with this case, please?  Thanks!

Intel bug reports are internal documents that are not accessible to the public.  You can though get a sense of some of the currently active RealSense related issues that have been officially logged as feature requests or possible bugs by looking at the list of public cases on this librealsense GitHub with an Enhancement label.
https://github.com/IntelRealSense/librealsense/labels/enhancement

Thanks for that info.  Though in looking though that list, I do have to take issue with labeling things like fixing a fatal exception as an "enhancement" rather than a "bug"!

Because GitHub cases associated with an internal report are kept open whilst that report is active, the Enhancement tag acts as a clear at-a-glance reminder to me not to close that case.  When an internal report is concluded, the Enhancement tag is removed from the GitHub issue paired to it so that issue can be considered for closure.

Hi @noirflux  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.