Global scripts not executed with @request

The script below contains a global region and a test region(only JavaScript with no request named test-1) . When executing test-1 region , the first global region cannot be executed. An exception will be raised during exec of the test-1 region.
{{@request
  exports.host = 'https://github.com';
  console.log('Global Script executed.');
}}

### test-1
GET {{host}} HTTP/1.1

But when the @request is removed, the test-1 regions will run successfully..
{{
  exports.host = 'https://github.com';
  console.log('Global Script executed.');
}}

### test-1
GET {{host}} HTTP/1.1

Maybe the variable replace hookitem should be moved after the @request region , just before the request execution?

@slackwareer The behavior is intentional. I only added the request hooks later. I wanted to offer an easy way to change/extend the replaced request again later. The difference between an execute and the request hook is basically the replacement.


Cluster stop

Aside from being slow, sawtooth cluster stop prints its Stopping: validator-*** messages all at once, without regard for when the validators actually stop. start does the same thing. Those messages have been moved to track the actual starting and stopping of validators.

@vaporos I agree, but I think this should go through now while Nick works on a better solution. The processes are getting killed anyways and this will save time for anyone using cluster.


Modify Extend Volume error message

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Solve Issue #268
Special notes for your reviewer:
Release note:

Coverage increased (+0.02%) to 59.292% when pulling c846a33e7dbae8a96d38b9dc239c97ffc00562c2 on PengYiOfOpenSDS:ExtendVolume20180206 into 9e2c4ce18eef28b1e103040ca952953f297d7ad9 on opensds:development.

Coverage increased (+0.02%) to 59.292% when pulling c846a33e7dbae8a96d38b9dc239c97ffc00562c2 on PengYiOfOpenSDS:ExtendVolume20180206 into 9e2c4ce18eef28b1e103040ca952953f297d7ad9 on opensds:development.

Coverage increased (+0.02%) to 59.292% when pulling c846a33e7dbae8a96d38b9dc239c97ffc00562c2 on PengYiOfOpenSDS:ExtendVolume20180206 into 9e2c4ce18eef28b1e103040ca952953f297d7ad9 on opensds:development.

Coverage increased (+0.02%) to 59.292% when pulling c846a33e7dbae8a96d38b9dc239c97ffc00562c2 on PengYiOfOpenSDS:ExtendVolume20180206 into 9e2c4ce18eef28b1e103040ca952953f297d7ad9 on opensds:development.

LGTM


Eye tracking recording not working in visualization demo scene

Describe the bug
Eye tracking recording / playback is not working in the  EyeTrackingDemo-05-Visualizer.unity scene
To reproduce
Steps to reproduce the behavior:

Open scene EyeTrackingDemo-00-RootScene.unity
Make sure to have MSBuild enabled in the project
Connect to Hololens 2 with Holographic Remoting
Play scene
With the menu in the scene, Navigate to EyeTrackingDemo-05-Visualizer scene
Press Record Button
Use eye gaze to generate heat map on left slate
Press stop Recording
Press Playback button
No playback is rendered

Your setup (please complete the following information)

Unity Version 2018.4.12f1
MRTK Version 2.3

Target platform (please complete the following information)

HoloLens 2

Did you set the capability to be able to write to the Pictures/VideoLibrary? Was this tested on the device or via Holographic Remoting?


Size of textfield box in which the user enters text

Is your feature request related to a problem? Please describe.
There seems to be a difference in the manner in which a text field is displayed as shown in the screenshot below and as it is shown in the swiftDialog wiki.
Describe the solution you'd like
I would prefer that the box that contains the entered text extend all the way to the left to the textfield label.
Describe alternatives you've considered
I changed the width of the dialog as a partial workaround.
Additional context
I carefully read all the documentation and looked at all issues. I hope I didn't miss anything. I have a 2023 Mac mini running the current versions of Sonoma and swiftDialog. Thanks for the great app--I  have already made extensive use of it in numerous scripts.

There seems to be a difference in the manner in which a text field is displayed as shown in the screenshot below and as it is shown in the swiftDialog wiki.

Can you link in the wiki where it it different to what you are expecting?

The link is:
https://github.com/swiftDialog/swiftDialog/wiki/Textfields
A screenshot:

ah - yeah, there is a maximum size set on the textfield size. If the window is small enough and the label is long enough then these will meet in the middle somewhere but for larger dialogs they will seperate. The label is left justified and the text box is right justified.

Thanks for the explanation.


Make MdcJsonProvider.writeMdcEntry() protected

Follow up of #957 to make the writeMdcEntry() method protected.
In my former proposal the intention for the protected on the method in question was to allow an easier way to write a custom extension by extending from MdcJsonProvider and only overwriting what one wants to override.
Though there's no obligation to do so for a custom handler, it would be nicer as it already provides useful functions that a custom MdcJsonProvider would need as well.
A dependency is there regardless if the methods are private or protected. However, opening up some of the (private) methods would make customer handlers easier as some of the inner framework methods are very hard to reach otherwise.
Having it private would enforce to copy a redundant version of some of the inner workings. Though from the perspective of the framework, the inner workings are often black-boxed. But from the perspective of a client writing a custom handler it's more cumbersome to rewrite part of the code to fulfill a similar or exact purpose.

It's not clear to me why logstash-logback-encoder needs to provide two extension points for customizing the writing of MDC entries:

override MdcJsonProvider.writeMdcEntry
AND
custom implementation of MdcEntryWriter

What can be done with overriding MdcJsonProvider.writeMdcEntry that can't be done with a custom implementation of MdcEntryWriter ?
For background, I prefer to keep the internal logic as encapsulated as possible to allow the library to evolve without constantly breaking compatibility for folks trying to reach into the guts of the library.  There are well defined extension points that are kept compatible  (In this case, JsonProvider and MdcEntryWriter).    MdcJsonProvider was not really meant to be extended.  In fact, I overlooked that the fields switched from private to protected in the previous change.  Now they can never change without breaking compatibility, which puts an additional maintenance burden on the library by making future changes more difficult.
I realize it's a tradeoff, because you might need to write a little bit more code, but that code will be much more resilient to future library changes if it sticks to the public interfaces.

@philsttr
philsttr> It's not clear to me why logstash-logback-encoder needs to provide two extension points for customizing the writing of MDC entries: 1. override MdcJsonProvider.writeMdcEntry AND 2. custom implementation of MdcEntryWriter [...]
philsttr> I overlooked that the fields switched from private to protected in the previous change. Now they can never change without breaking compatibility [...]
Sorry, the private-to-protected change was the 1st commit in the other pull request (https://github.com/logfellow/logstash-logback-encoder/pull/957/commits/178b29ee791494a32c36d6910fb29f79a5be2f39).
It was trying to open an easy way to some of the inner methods before I came up with the idea of the MDC entry writers, so that a new custom MdcJsonProvider was possible without having to rewrite/copy the writeTo()-function only manipulating the way of the output.
I have to admit I'm not used to write or contribute to frameworks (yet).
Wouldn't it be possible to directly follow-up with a change back to private again (as it was an "accident")? If it comes shortly after the last release, clients tend to use the higher available release version ... so then not having much time to use the "exposed" protected methods yet.
philsttr> For background, I prefer to keep the internal logic as encapsulated as possible to allow the library to evolve without constantly breaking compatibility for folks trying to reach into the guts of the library.
Sure, that's natural ... I think I can quote from "Clean Code" ;-)
Clean Code, chapter 8 "Using Third-Party Code"> "There is a natural tension between the provider of an interface and the user of an interface. [...] Users [...] want an interface that is focused on their particular needs."
As you've said it's a trade-off, but also a matter of personal preference focusing on ones own need more than the others point of view, in the tension field between framework provider and framework user.
I can certainly also see your point.
I suggest to change the modifiers back to private if you prefer, just wanted to give the background thinking behind it.

Closing this in favor of using a custom MdcEntryWriter


linter fix part 2

@maniarathi PTAL when you get a chance! Thanks a lot!

Awesome thanks so much, @maniarathi !! Merging 😄 🚀 !


To do list checkbox issue

Hello, I found that the html text obtained after the check box is checked is still unchecked html text, so I cannot save the checked state.

The same to me.How do you do that?


fix: set the correct rpc response type when meet an error

Go runner always return the request rpc type as response type, according to the protocol, it should return RPCError when meet an error.

Let's merge the master branch to run E2E Test.

Codecov Report

Merging #102 (71906e7) into master (bd5d11f) will decrease coverage by 1.74%.
The diff coverage is 27.77%.

@@            Coverage Diff             @@
##           master     #102      +/-   ##
==========================================
- Coverage   82.38%   80.64%   -1.75%
==========================================
  Files          14       14
  Lines         897      899       +2
==========================================
- Hits          739      725      -14
- Misses        113      132      +19
+ Partials       45       42       -3

Impacted Files
Coverage Δ

internal/server/server.go
53.90% <27.77%> (-11.97%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Fix Java-8 Bug

Current coverage is 63.13%

Merging #57 into master will increase coverage by +0.53% as of 2d9e61b

@@            master     #57   diff @@
======================================
  Files           26      26
  Stmts          730     830   +100
  Branches        90      92     +2
  Methods          0       0
======================================
+ Hit            457     524    +67
  Partial         36      36
- Missed         237     270    +33

Review entire Coverage Diff as of 2d9e61b

Powered by Codecov. Updated on successful CI builds.

Just started reviewing :) :octocat: Sent from GH.

Just started reviewing :) :octocat: Sent from GH.


Update to Rails 5.2.0

Rails 5.2.0 will give us ActiveStorage and all sorts of other goodies:
http://edgeguides.rubyonrails.org/5_2_release_notes.html
rails/rails#30016 introduced Rails::AppBuilder#ruby_version to create
a .ruby_version file. To avoid getting stuck in an overwrite prompt
when we try to create our own .ruby_version file based on
Suspenders::RUBY_VERSION, I overrode #ruby_version.
Rails 5.2.0 adds bootsnap to the Gemfile and to config/boot.rb. I added it
to our Gemfile, but we could also skip_bootsnap by default if we don't
want to use it.

Looks like you're missing the addition of a few lines for bootsnap to work:
https://github.com/Shopify/bootsnap#usage

@toobulkeh Rails 5.2 adds require 'bootsnap/setup' to config/boot.rb by default. Is there something else I am missing?

Merged as d3c2e89f91784f60b2923b947114245a3c427a82, thank you!


(Do not merge yet) Add CleanRL to Hub Documentation

⚠ Do not merge for now (I opened a PR to visualize the result with doc builder) ⚠
Costa worked on a new CleanRL integration for the Hub: https://github.com/vwxyzjn/cleanrl/pull/292
In order to help CleanRL team to integrate it, I worked on the documentation on the Hub and the moon-landing integration.
For now since we don't have the final version of the integration, upload and download your models parts of the doc are not defined yet.
TODO:
🔲 Add in Librairies.ts the method to load_from_hub
🔲 Add track in Elastic.ts
🔲 Update the library with load_from_hub and push_to_hub
TODO From CleanRL PR:
🔲 Add in model card generation metadata["library_name"] = "cleanrl"

You can use the PR "draft" mode if it's not ready to merge yet 🙂

@simoninithomas this seems quite outdated. Would you like to update it and merge it or should we close? Note: the JavaScript part of the code now lives in https://github.com/huggingface/huggingface.js

We can close it. I'm closing it.


Help on decoding Model with 'foreign key to same table' when using table splitting

I am building a tool that can compare the DbContext's Model with a database (I get the database model via the design-time scaffolder). The idea is to to provide feedback to the user when the DbContext's Model and the database don't match - rather like a user-friendly version of EF6.x's context.Database.CompatibleWithModel, with human-readable error messages.
I have got it to work with pretty much every combination of table mapping, but I have run into problems with table splitting. Basically the context.Model says there is a foreign key to the table, but because its the same table it doesn't exist. NOTE: owned types do the same, but I was able to find/fix that.
Obviously EF Core works out that a foreign key isn't needed, but I couldn't find how to detect that in the Model.
Steps to reproduce
My code is quite large, and I have given you the section that goes wrong out of the Stage1Comparer class (found here). The bit I want to fix is the method IgnoreForeignKeyIfInSameTable at the bottom.
The whole project is an open-source on Github at JonPSmith/EfCore.TestSupport and on Nuget as EfCore.TestSupport. There is some documentation in the wiki
private void CompareForeignKeys(CompareLog log, IEntityType entityType, DatabaseTable table)
{
    var fKeyDict = table.ForeignKeys.ToDictionary(x => x.Name);

    foreach (var entityFKey in entityType.GetForeignKeys())
    {
        var entityFKeyprops = entityFKey.Properties;
        var constraintName = entityFKey.Relational().Name;
        var logger = new CompareLogger(CompareType.ForeignKey, constraintName, log.SubLogs, _ignoreList, () => _hasErrors = true);
        if (IgnoreForeignKeyIfInSameTable(entityType, table))
            continue;
        if (fKeyDict.ContainsKey(constraintName))
        {
            //Now check every foreign key
            var error = false;
            var thisKeyCols = fKeyDict[constraintName].Columns.ToDictionary(x => x.Name);
            foreach (var fKeyProp in entityFKeyprops)
            {
                var pRel = fKeyProp.Relational();
                if (!thisKeyCols.ContainsKey(pRel.ColumnName))
                {
                    logger.NotInDatabase(pRel.ColumnName);
                    error = true;
                }
            }
            error |= logger.CheckDifferent(entityFKey.DeleteBehavior.ToString(),
                fKeyDict[constraintName].OnDelete.ConvertReferentialActionToDeleteBehavior(entityFKey.DeleteBehavior),
                    CompareAttributes.DeleteBehaviour);
            if (!error)
                logger.MarkAsOk(constraintName);
        }
        else
        {
            logger.NotInDatabase(constraintName, CompareAttributes.ConstraintName);
        }
    }
}

private bool IgnoreForeignKeyIfInSameTable(IEntityType entityType, DatabaseTable table)
{
    if (entityType.DefiningEntityType == null ||
        entityType.DefiningEntityType.Relational().TableName != table.Name)
        //if not a owned table, or the owned table has its own table then carry on
        return false;

    //It is a foreign key that links back to the same entity, so we ignore
    return true;
}

The unit test that fails is CompareSpecializedDbContext fact in CompareSpecialised class. The database has a BookDetails and BookSummary class which do the table splitting. (Ignore the second error - that is due to issue #9652)
[Fact]
public void CompareSpecializedDbContext()
{
    //SETUP
    using (var context = new SpecializedDbContext(_options))
    {
        var comparer = new CompareEfSql();

        //ATTEMPT
        var hasErrors = comparer.CompareEfWithDb(context);

        //VERIFY
        hasErrors.ShouldBeTrue();
        var errors = CompareLog.ListAllErrors(comparer.Logs).ToList();
        errors.Count.ShouldEqual(2);
        errors[0].ShouldEqual(
            "NOT IN DATABASE: BookDetail->ForeignKey 'FK_Books_Books_BookSummaryId', constraint name. Expected = FK_Books_Books_BookSummaryId");
        errors[1].ShouldEqual(
            "DIFFERENT: BookSummary->Property 'BookSummaryId', value generated. Expected = OnAdd, found = Never");
    }
}

Further technical details
EF Core version: 2.0.0
Database Provider: Microsoft.EntityFrameworkCore.SqlServer (and others)
Operating system: Windows 10
IDE: Visual Studio 2017

@JonPSmith Can you explain a bit what you mean by, "Basically the context.Model says there is a foreign key to the table, but because its the same table it doesn't exist?" Are you saying that EF is not creating FK constraints in the database for this case?

Hi @ajcvickers,
In getting the information to explain the problem I found a way around it, so this isn't a problem any more (I have updated the original code with my fix).
However, the basic problem is that foreign key information is incorrect in the the context.Model for table splitting classes. I have worked around the problem, so you can close the issue, but I thought you might like to know what is going on.
The incorrect information in the context.Model
For table splitting the context.Model incorrectly says that there is one foreign key between the two classes involved in the table splitting  arrangement, i.e. it has a foreign key that points to the same table. When EF Core creates the table it doesn't include that foreign key. The code below shows my set up.
I have two tables that are involved in table splitting, BookSummary and BookDetail, which both map to the table called Books. Here are the entity classes and the configuration code.
public class BookSummary
{
    public int BookSummaryId { get; set; }
    public string Title { get; set; }
    public string AuthorsString { get; set; }
    public BookDetail Details { get; set; }
}

public class BookDetail
{
    public int BookDetailId { get; set; }
    public string Description { get; set; }
    public decimal Price { get; set; }
}

The configuration code is
public void Configure
    (EntityTypeBuilder<BookSummary> entity)
{
    entity.HasKey(p => p.BookSummaryId);

    entity
        .HasOne(e => e.Details).WithOne()
        .HasForeignKey<BookDetail>(e => e.BookDetailId);
    entity.ToTable("Books");
}

and
public void Configure
    (EntityTypeBuilder<BookDetail> entity)
{
    entity.ToTable("Books");
}

The actual SQL table looks like this, i.e. there is no foreign key.
Executed DbCommand (4ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
CREATE TABLE [Books] (
    [BookSummaryId] int NOT NULL,
    [Description] nvarchar(max) NULL,
    [Price] decimal(18, 2) NOT NULL,
    [AuthorsString] nvarchar(max) NULL,
    [Title] nvarchar(max) NULL,
    CONSTRAINT [PK_Books] PRIMARY KEY ([BookSummaryId])
);

If I look at the context.Model.GetEntityTypes() and look at the BookDetails entityType then the entityType.GetForeignKeys() returns one foreign key, which is wrong because the foreign key points to the same table.
Note: Owned types have the same problem (i.e., an incorrect foreign key in the context.Model) and also defines a primary key, which is also incorrect. These were much easier to work around because of the entityType.DefiningEntityType property.

It is correct model.
At core level, you have 2 entity types with a unique one to one relationship defined with them. From perspective of Core, every entity type has their own storage. The ToTable calls sets a relational annotation on EntityType indicating what should be the table at "relational" level. This is by design. Relational has concept of table splitting but non-relational providers do not have it that way. (It would map with nested json more than table splitting). Hence when you look at model in core, yes there is foreign key defined.
The code which makes FK go away is in relational and multi step.

SharedTableConvention - This convention configures owned entity to map to same table as the owner entity. It also unique-fy column names for sharing the table.
RelationalPropertyAnnotations.GetDefaultColumnName() maps column name of dependent PK (which is also FK to principal PK. (In above case it will map BookDetailId to BookSummaryId column.
TableMapping class in migration which is the final key to swallow the FK.

https://github.com/aspnet/EntityFrameworkCore/blob/c7c7d455ed9877f56164e6531d23fedffa104dc7/src/EFCore.Relational/Metadata/Internal/TableMapping.cs#L77-L83
If you look at TableMapping.GetForeignKeys() method, what it does is, finds all FK defined on all entityTypes and removes FK which are pointing to the same table & where column names for FK properties are same as principal columns (set in step 2 above). For table splitting case, the FK is between PK to PK and we just use 1 column to store value and using implicit relationship rather than creating an FK. This is the place where FK from core gets swallowed by Migrations because it is unnecessary to create.
Since the whole concept of table sharing and mapping is relational, if you are just looking at core metadata it will not match your database table till you apply relational level transformations to it.
In your case the diffing method,
For dependent shared table of (BookDetail), you need to find all FKs defined on the PK of it which are pointing to entityType (BookSummary) being stored in same table (Books) and uses the same column name for FK properties as Principal properties (BookSummaryId). Such FKs will be swallowed and ignored. Your updated check is almost true but you also need to do structural comparison over FK properties for their column names.

Thanks @smitpatel for taking the time to provide me with a very detailed answer. Excellent stuff. I will look at that and make some changes.


Recursive file watcher

Builds on fsnotify by adding recursive dir watching with debounce.
Used in https://github.com/istio/operator/pull/29.

Thanks for the review and even more so testing it out and discovering the problem.
What surprises me about your experience is that this should be a dir (rather than file) watcher so in principle if a new file is created, we should see fsnotify.Write event. I wonder if you could just make a simple mod on line 96 of also checking for delete and create events? Maybe the dir is just as OR over all file events under it. If so it would not be intuitive but at least I guess it will solve the problem you mentioned.
If that doesn't fix it I think we can create an issue - it's perfect for someone else in the community to take on.

It seems that an fsnotify watch on a dir is simply a watch on all of the files in that subtree.
So if a file is added it's not a WRITE on the directory, just a CREATE on the file itself. Same with REMOVE.
If we want to be notified about file creates and removes, than we should add those events as well.


upgrade to wagtail 1.5.3 and overlay bug fix for background colour

Coverage increased (+0.06%) to 69.715% when pulling 5f287329c905032d2b9a99a9675b9da9fd757bdf on master into 19f01956503c4882b5b23bd72fd9486e3480c55e on production.


refactor (tracing): move from github.com/pkg/errors to 'errors' and 'fmt'

The package https://github.com/pkg/errors is not maintained anymore.
This PR replaces https://github.com/pkg/errors with official errors and fmt golang packages in tracing package
Signed-off-by: Matthieu MOREL<EMAIL_ADDRESS>

Thanks!


segfault

hello,
I saw your project and would love it to work, that would be awesome to drop windows entirely
that said, I cloned the project, enter 'python garmin.py' as root (libusb needs root acces it seems, maybe a good udev rule would be enough to run as a "normal" user)
here is the output :
root@benbook:/home/lotsob/Garmin-Forerunner-610-Extractor# python garmin.py
configval
1
0,0
129
1
Segmentation fault
here is the dmesg output :
[1249078.768222] usb 2-1: new full speed USB device number 98 using uhci_hcd
[1249078.935525] usb 2-1: New USB device found, idVendor=0fcf, idProduct=1008
[1249078.935539] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[1249078.935550] usb 2-1: Product: ANT USBStick2
[1249078.935558] usb 2-1: Manufacturer: Dynastream Innovations
[1249078.935567] usb 2-1: SerialNumber: 104
[1249574.212217] usb 2-1: reset full speed USB device number 98 using uhci_hcd
[1249574.363320] python[10930]: segfault at 24 ip b725f6be sp bffdde30 error 4 in libusb-1.0.so.0.1.0[b7255000+e000]
I would love to contribute more, tell me what would help :)
cheers

Your issue is probably caused by PyUSB 1.0 (seems to be a bug which makes it segfaults on version before Alpha 2). It is recommended to use pip to install PyUSB:
pip install pyusb
I've also updated the README file to indicate this. It would be great if you can test this and see if it works :).

Yeah, that is how the script ends right now. So it looks like it should have transferred all of your data. Right now it just tries the next item from the file index, which is last + 1, which obviously doesn't exists, hence the exception.
The number 125 is the size in bytes of the latest file transferred, not the file count, so you are all good.
If you like you can open three new issues for these problems so we have a better way of tracking them.
 There is another bug (thing not done) concerning those files. They are probably padded with some null characters. I'll try to fix that soon.


Configure Cake to generate releases and nugets

Working locally. Now is time to make it happen on AppVeyor


Ignore docker mapper storage in spec as well

Docker mapper storages are ignored in collecting the metrics (ref: #58), but the specs are still collected. The information on docker mapper storages in the host page are not that useful. How about ignoring the docker storages?

LGTM

Thank you.


bundle jsdom for window mocking

This is the final step to make running this polyfill in node a seemless experience. Previously, you would have to supply your own window implementation (jsdom) if you wanted to consume this. After this change, you get jsdom's window mocking for free, and all you have to do to use this is require('webvr-polyfill')

This would increase the size of browserify/webpack includes (which would use src/node-entry) from 244k to 5.1M (!), in addition to possibly conflicting (duplicate) versions of jsdom for a node app using jsdom. I think the solution to this is related to #47, and think it should take a window object (defaulting to window), where a jsdom window could be provided.
Providing alternative DOMs is out of the scope of this project, especially with the code size increase in browser scenarios, but supporting alternative DOMs, via passing in a global, should be supported. Would that cover your use case @kellyselden ?

I'm a little confused. The browserify bundler wouldn't use the src/node-entry file would it? The build script uses browserify src/main.js as the entry point. So I don't see how the bundle size would explode like that, unless browserify automatically bundles everything in npm dependencies?

Correct, but if aframe or someone else using this for example calls in require('webvr-polyfill'), THEIR browserify/webpack/loader will call whatever is in main in the package.json and build the polyfill, rather than use the build/webvr-polyfill.js due to #238

Ah you're right. I'm a little rusty on bundlers.

@kellyselden would exposing the polyfill to accept a window/jsdom object work for your use case? Something like new WebVRPolyfill(window, config)?

Yes it would.


Import show views from old YACS admin app

Since we have the schools and departments working, it would be good to have a show view implemented. The old show view can be reused. Add unit and end-to-end tests where appropriate.

Ideally, this will be combined with the form.

We're switching to a much simpler UI to minimize the number of pages the admin has to sift through (blog post to come later).


Speech in React Component

Hi,
I just implemented BotFramework-WebChat react component with custom build. I want to implement the speech functionality also. As I see it is available in the web-chat sourcebut I didn't find any way to implement this in react. I found there is no export for Cognitive service. Can you please suggest the best way to implement?
Thanks :)

@shahidkhuram can you provide a React-based example?

In our <script src="..."> scenario, it works because the user would implicitly include 2 entrypoints like this:
<script src="../../botchat.js"></script>
<script src="../../CognitiveServices.js"></script>

But in React scenario, our entrypoint is BotChat.ts. But unfortunately, BotChat.js does not export CognitiveServices/SpeechRecognition.ts and CognitiveServices/SpeechSynthesis.ts. Thus, CognitiveServices/* was not able to use under React.

Hello, any news for this? It is possible to implement the speech capabilities with a React component?

@HernanCote I think the current packaging approach is not correct and please bear with us that we need time to fix that. I think we should:

Publish botframework-webchat without CognitiveServices.js
Publish a new package botframework-congitiveservices for the CognitiveServices.js (or rename it to BingSpeech.js)

On top of that, I would also love to see some refactoring on CognitiveServices.js. For example, making the CognitiveServices.js to conform to Web Speech API. We are already using Web Speech API in Web Chat (via BrowserSpeechRecognition class) and I think it is easy to do that.
And then we can move to Lerna to package both botframework-webchat and botframework-cognitiveservices in the same GitHub repository.

Sorry for late reply, but I fixed it by myself to use of CognitiveServices. I used the local version of this package. I just clone this packge and added the following line in Botchat.ts
export * from './CognitiveServices/lib';
and then I make the local npm package by compiling it locally to use in my project.
After that I added this package to my project and  below is the implementation
import { Chat, SpeechSynthesizer, SpeechRecognizer, SynthesisGender, DirectLine } from 'botframework-webchat';

const speechOptions = {
    speechRecognizer: new SpeechRecognizer({ subscriptionKey: 'YOUR_COGNITIVE_SPEECH_API_KEY' }),
    speechSynthesizer: new SpeechSynthesizer({
        gender: SynthesisGender.Female,
        subscriptionKey: 'YOUR_COGNITIVE_SPEECH_API_KEY',
        voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'
    })
};

I hope this could help you.

That's exactly what I was thinking, making another NPM package only for the Cognitive Services.
Wished we can do better to unblock you sooner.


Import errors in node

TypeError: prelude_1.prop is not a function

I tried using Maybe's for the first time in my life, and wanted to do a simple console.log. I got a similar result with ap via @typed/prelude:
https://github.com/marcusnielsen/mostjs-labs/blob/master/src/index.ts#L11
 const log = (maybe) => prelude_1.ap(prelude_1.toMaybe(console.log), maybe);
TypeError: prelude_1.ap is not a function

I was guessing it's my fault since I've never used ap, Maybe, or this framework before. But seeming this issue, maybe it's the same bug? Any advice for moving forward?

Hey @marcusnielsen there seems to be some sort of issue with the way TypeScript compiles ES2015 modules to commonjs. If you're interested in Maybe in particular, I'd suggest using @typed/maybe directly until we can figure out this issue here.

Additionally if you're using something like webpack to bundle your application, you can add module to resolve.mainFields to use the ES2015 build instead and avoid this bug.

Using @typed/maybe worked out well! (I'm using tsc without webpack.)
Thanks for the help, I just did my first Maybe and ap , and it feels awesome 🎉

I'm thinking that going forward we could just have yarn add @typed/prelude be a convenient shortcut for yarn add @typed/either @typed/functions @typed/lenses @typed/list @typed/logic @typed/math @typed/maybe @typed/objects @typed/strings but not re-export these packages from itself, but instead requiring explicit imports from the other @typed/* packages.
What do you think?


 Which double click being fired by each single touch click on the viewer.
As shown in the video, I attached a simple listener to the document to print ( hello dblclick ) if a double clicks triggered.
(0:00 to (0:27) Just demonstrated mouse double click works fine.
At time (0:27+) I double clicked using touch on the shuttle ring (HTML element) bottom left of cesium page, and then I single touch clicked on the viewer which then you will see on console that ( hello dbclick ) being called for every single touch click on viewer.
This bug could be also initiated on any HTML component like ( e.g. buttons in toolbars ) when using touch screens.
Tested on Chrome.

Hmm, very interesting.  Thanks for including the video @Alakeel!  I'm not sure if this is a Cesium bug, or a browser bug.  We'll have to look into it further.  Could you try a different browser and let me know if you see the same thing?  Thanks!

Hmm, very interesting. Thanks for including the video @Alakeel! I'm not sure if this is a Cesium bug, or a browser bug. We'll have to look into it further. Could you try a different browser and let me know if you see the same thing? Thanks!

@hpinkos I have tested this on ( Internet Explorer , Firefox ) and I don't see the same thing. This issue happens only on Chrome when using touch monitors.
I'm not very sure if it's Cesium or Chrome Browser bug. I tried to make a demo page with an empty canvas element and I don't see this issue happens in Chrome. The issue exists somewhere on cesiumContainer, it needs to be investigated.

Great, thanks for the extra info @Alakeel!  We probably won't have time to look into this very soon, but if you figure something out we'd be more than happy to review a pull request =)


[BUG] bloxstrap freeze for 10 sec.

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.
[X] I have thoroughly looked through the available Wiki articles and could not find a solution to my problem.
[X] I am using the latest version of Bloxstrap.
[ ] I did not answer truthfully to all the above checkboxes.

What problem did you encounter?
my bloxstrap just freeze for 10 sec. i already set the framerate to zero at the bloxstrap menu, but didin't worked. any suggestion?
Bloxstrap Log
N/A

Is it Roblox or Bloxstrap freezing?

it's the Bloxstrap. i've tried use the Roblox app. and it's fine.

Can you send your log file?

too late... i already uninstall the apps. is it okay if i download it again and play it?

If you are using Vulkan, remove it.

too late... i already uninstall the apps. is it okay if i download it again and play it?

Yes, do that please.

Issue has stale for over a month, and will be reopened if further information is given.


Performance is poor after re-train a Mask RCNN

Hi,
I took the instance-level pseudo labels generated by running `make_ins_seg_labels.py' and kept the instance mask whose score is higher than 0.
Then, I transfered these labels from *.npy to cocostyle json annotation and trained the standard Mask R-CNN with ResNet-50-FPN.
However, the performance I've get is:

Specifically, box mAP of AP50 is 45.8, segmentation mAP of AP50 is 22.6.
I noticed that the instance number in pseudo label is about 2/3 of the gt instance number for `train_aug' set.
Did I miss something to reimplement the performance of Mask R-CNN with pseudo label?
Thanks a lot!

Hi @bityangke,
It is hard to give you a straight-forward answer to your question. Coco mask annotation is composed of polygons, not pixel-wise instance labels. Hence, there are many ways of transfering the psuedo labels for mask rcnn. Please make sure you did not make any mistakes during the conversion process.
Plus, the mask rcnn should be pretrained on imagenet.

Thanks very much！

@bityangke did you fix this issue? do you have a code for label transformations? thanks


Acceptance test for archiving into ENA

Any dataset currently in Valid state, should be able to be archived into ENA.
When a developer has archived 3 datasets without any issues, this task can be reassigned to a wrangler.
We can consider this task done when 10 datasets have made it into ENA staging.

Amnon followed the SOP for archiving and apparently the process completed successfully
However the logs contain errors which are not visible to the user which is a problem

@amnonkhen to give an update on where we are - archiving is not working on dev.


check root on Linux and restart script as root

Wouldn't it just be better if you prevented execution unless $EUID = 0 and uname returns Linux? Just dump a message "palera1n requires root privileges on Linux to access your device".

Wouldn't it be better if you prevented execution if $EUID != 0 and uname returns Linux? Just dump a message "palera1n requires root privileges on Linux to access your device".

it would be more efficient this way

Wouldn't it be better if you prevented execution if $EUID != 0 and uname returns Linux? Just dump a message "palera1n requires root privileges on Linux to access your device".

it would be more efficient this way

In my opinion, informing the user of the proper usage is more proper. Another gripe with this is that it requires yet another dependency, 'caffeinate'. I do see why it has been added, but it just all seems really complicated and it adds yet another external dependency.
Is it really that bad for the UX to be like this:

User doesn't read docs, executes palera1n without root privileges on Linux.
Palera1n informs the user that it needs root privileges on Linux.
User does sudo !!
User in the future now knows to execute palera1n with root privileges on Linux.


[CI] New CI system

Issue #, if available:
Description of changes:
This PR brings the new CI system based on GitHub Action + AWS Batch. For an example of what it looks like: https://github.com/yinweisu/autogluon/actions/runs/2571378102
The new system

can save us from maintaining our own machine.
is stateless between runs. (a big trouble with our current CI)
is more secured as it is approved by security review.

One tradeoff is that the new system is a little bit tricky in terms of updating because of security concerns. For more detail, refer to .github/README.md
The final docs are built but not written to our bucket for now with the new CI system. The plan is to run both CI systems in parallel for a while to ensure the stability of the new system. Once we think it's stable, we can get rid of Jenkins :).

Also re-enabled hpo tests as it runs correctly on the new system.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-1907-2 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1907/2/index.html

Job PR-1907-3 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1907/3/index.html

Job PR-1907-4 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1907/4/index.html

Job PR-1907-6 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1907/6/index.html


New contributions

A few new things here, all having to do with MS quantification, which @spoeka22 and @AlexanderKrabbe contributed to the user_ready branch while @KennethNielsen and I worked on #30:

functionality in MSInlet.gas_flux_calibration for calibration with dilute calibration gas
an effective capillary length calculator in MSInlet
CO2 properties added to constants.py

It's been tested against my most advanced EC-MS calibration script, https://github.com/ScottSoren/pyCOox_public/blob/ixdat_v0p2/paper_I_fig_S1/paper_I_fig_S1.py
I think we can close the user_ready branch when this is merged to main!

I will tidy up the docstring in a new pull request when I figure out my git again.
On Wed, 16 Mar 2022, at 21:02, Kenneth Nielsen wrote:

@.**** approved this pull request.
A few minor docstring nitpicks. Otherwise good to go from me. I'll set to approve as they are so small.
In src/ixdat/techniques/ms.py https://github.com/ixdat/ixdat/pull/44#discussion_r828242402:

     self.w_cap = w_cap

     self.h_cap = h_cap
     self.p = p
     self.T = T
     self.gas = gas  # TODO: Gas mixture class. This must be a pure gas now.

def calc_l_cap_eff(

   self, n_dot_measured, gas=None, w_cap=None, h_cap=None,  T=None, p=None):

   """Calculate gas specific effective length of the capillary in [m]

       and add {gas:value} to l_cap_eff (dict)

The second head line should only be indented to be under the first "
In src/ixdat/techniques/ms.py https://github.com/ixdat/ixdat/pull/44#discussion_r828396805:

     self.w_cap = w_cap

     self.h_cap = h_cap
     self.p = p
     self.T = T
     self.gas = gas  # TODO: Gas mixture class. This must be a pure gas now.

def calc_l_cap_eff(

   self, n_dot_measured, gas=None, w_cap=None, h_cap=None,  T=None, p=None):

   """Calculate gas specific effective length of the capillary in [m]

       and add {gas:value} to l_cap_eff (dict)

   Args:

       w_cap (float): capillary width [m], defaults to self.w_cap

Capitals after colon. There are a few of these
In src/ixdat/techniques/ms.py https://github.com/ixdat/ixdat/pull/44#discussion_r828396988:

       gas (dict or str): the gas in the chip, defaults to self.gas

       T (float): Temperature [K], if to be updated

       p (float): pressure [Pa], if to be updated

   Returns:

       float: gas specific effective length in [m]

   """

   n_dot_predicted = self.calc_n_dot_0(gas=gas, w_cap=w_cap, h_cap=h_cap, T=T, p=p)

   l_cap_gas_specific_eff = self.l_cap * n_dot_predicted / n_dot_measured

   self.l_cap_eff[gas] = l_cap_gas_specific_eff #add effective l_cap for specific gas

   return l_cap_gas_specific_eff

def update_l_cap(self, gases=[]):

   """ Update self.l_cap from average of values in dict l_cap_eff

No space before "update"
—
Reply to this email directly, view it on GitHub https://github.com/ixdat/ixdat/pull/44#pullrequestreview-911974809, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJT4LSLP4BI4OPVVVYHGDZ3VAI45JANCNFSM5Q3ONJZA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>

@AlexanderKrabbe that is a little fun since Scott kind of submitted this one for you in this extra round. The branch that we are merging is owned by this main repository, which you don't have push access for. If you want to fix it, what I would suggest is that you make yourself a new feature branch, of this one ixdat/ixdat:new_contributions fix stuff there and then ask Scott to merge the fixes into this branch.

@KennethNielsen ahhh. smart :-)

@AlexanderKrabbe I hope so. Let me know if the git gives you gripes.

@AlexanderKrabbe addressed the comments in #48. Merging!


Revamp hir_def attribute API

This adds AttrsWithOwner, which can construct an accurate AttrSourceMap without requiring additional information from the caller.
r? @Veykril

This looks a lot better
bors r+

changelog internal


Add pagination and streaming

Close #67

Coverage increased (+2.4%) to 86.702% when pulling c57578df2f46215fc171ee884a8c2edf2726f3c5 on jeffweiss:add_pagination into f954f84f72d508116e8d03f1bfceea020c2f138e on edgurgel:master.

Hmm.. I'm using URI.to_string, which is only available in Elixir 1.1+. @edgurgel, @duksis, thoughts on bumping minimum Elixir version to 1.1?

Also just saw #69 merged, I should update to use those options as well. Standby...

Coverage increased (+2.4%) to 86.702% when pulling 8de890f3e52b21b15d5d9d6a58b6ca198097d7f1 on jeffweiss:add_pagination into f954f84f72d508116e8d03f1bfceea020c2f138e on edgurgel:master.

first of all - thanks for the amazing contribution @jeffweiss !
In regards of bumping required Elixir version, I would prefer backporting URI.to_string for Elixir 1.0 than dropping the support for it.
What's your take on this @edgurgel ?

I agree @duksis. Let's do this in 2 stages.

Oh, we might not need to backport anything. It looks like String.Chars.to_string(URI) was defined in 1.0 even though URI.to_string was not. This might be a very simple change.

Coverage increased (+2.4%) to 86.702% when pulling b6c9c55372182fe88241f4b706b2f49db5ae5ae6 on jeffweiss:add_pagination into f954f84f72d508116e8d03f1bfceea020c2f138e on edgurgel:master.

\o/  --- If you have other changes you'd like me to make lemme know.

thanks! We will let you know when other changes will be needed. :wink:

:+1: We should release a new version with this contribution. I should give owner access yo you duksis. Do you have an e-mail registered with hex so I can add you? :)

@edgurgel should be<EMAIL_ADDRESS>

$ mix hex.owner add tentacat<EMAIL_ADDRESS>Adding owner<EMAIL_ADDRESS>to tentacat

:tada:


Fix handling of SharedDirectory ops on subdirs created on a detached data store

Description
SharedDirectory was using -1 all the time for locally created subdirectories.
This leads to bugs when considering the logic in isMessageForCurrentInstanceOfSubDirectory when the SharedDirectory is additionally detached: no ack op will ever be received for that subdir (since an op is never sent), but clients who load using the attachment summary may start sending ops which refers to the same subdirectory.
merge-tree has a similar problem which is resolved by using -1 or 0 depending on the attach state of the DDS, see UnassignedSequenceNumber vs UniversalSequenceNumber. The fix here brings SharedDirectory handling closer to merge-tree's.

do we need to port this back to 5.0 and maybe 4.0

do we need to port this back to 5.0 and maybe 4.0

yeah, I think so. I wanted to audit other places that -1 gets used in the code to see if there are any more bugs of the same variety before doing so, but prepped this fix early as it should be enough to fix the immediate issue (making some test packages right now)


[Feature] Support optional rotation for non line labels like city, area, country etc.

Since

fixed by c42c0ee21f67ec7fc0808000c8507aeb85dc6650

fixed by 17c50b59ba669c80f51af4c09e98d0fedfa4e4bf


Added categories to extras sidebar

I've split the extras components into categories based on the drei categories and ordered alphabetically.
Addresses this issue:
https://github.com/threlte/threlte/issues/708

@edsunman thank you for separating this into a distinct PR! I'm gonna have a look at it at some point this week (its been a busy one!).

I generally really like the direction you've taken here, but there are some specific issues I'd like to address.
The main issues is a mix of domain- and type-based categories. I think we should get rid of hooks, lifecycle and plugins as distinct categories: their contents should be distributed in the other categories. If necessary, new categories can be added, fitting the domain orientated approach the other categories have. As a concrete example of benefits, I would like to see all suspense related stuff together, and some of the audio hooks with the audio components.
The lesser issues relate to problematic "et cetra" categories: abstractions and misc. This is always a challenge with categorization, but maybe we can sort out a good solution. I really like the staging category but I'm conflicted since it would be nice to have shadows together.
I'm wondering if we should have a content category: <HTML>, <Text>, <Text3DGeometry> and the upcoming <Image> could all go there.
Also, while I like the specificity of the shaders category, I wonder if its a bit too narrow. In my version I had a category called effects, intentionally decouple from implementation details. If I was browsing extras, I would put for example <Edges> in the same category mentally, since I would be looking for cool effects to augment my models.
This is all very loose feedback, and nonbinding. I don't think this kind of thing has a clear answer. I'd love to also hear the perspectives of the rest of the team.

@ixxie thanks for the feedback! I agree with removing the sub-categories and just using domain based categories. I felt it was a bit messy with the components listed with angle brackets < >. What do you think about removing the angle brackets in the sidebar but keeping them in the title?
https://github.com/threlte/threlte/assets/14927523/a00ade00-2226-4e19-8b1e-a54c2486ebff
I don't have any strong opinions on what the categories are called so happy to go with what you suggest!

The problem is that all the other docs have the <AngledBrackets> notation. I personally like it, but I'm open to a change. However, changing this in all the docs is maybe something beyond the scope of this PR and should be discussed with @grischaerbe maybe.
The content category in the video now also combined you previous staging category. I actually liked that category as it was! Maybe you can separate them? If you could then deploy that I could have a look.

Good point about keeping the brackets to match the rest of the docs. I've put them back in and deployed so you can take a look.
Do you think the 'content' could be renamed to 'objects' to make it a bit more specific?

Looks better!
Regarding content - I meant the include the kind of stuff that lets you put text and images in the scene. I was thinking that <Environment>, <Stars>, <Sky> and <Grid> would be more like staging stuff.
Regarding effects and shaders I'm somehow not entirely happy with the name. I also realized calling it merely effects is going to be very confusing in Svelte 5. How about special effects? I'm open to other ideas...
I really would like to have all suspense related stuff contiguous in the menu. So either we break the alphabetical ordering rule just for this, or we split the category. What do you think?
 What was your reasoning to put them there? It may be that the concept of "staging" is very specific and I don't get it, but I was thinking of stuff you put on a stage (like theater set design).

I created a google doc so it's easy to discuss and quickly move things around without pushing lots of changes:
https://docs.google.com/document/d/1PPylp7XjcJS9HElB5jT7r-kPhuxTVcjCbzNi8OSQwZw/edit?usp=sharing
Staging is quite a broad term that can refer to the set dressing but also to how actors are positioned and move around the stage. So I was thinking that content would refer to the objects in the scene and staging would refer to how those things are lit and positioned, with things like <Float> , <Billboard> etc positioning the objects. I think you are right that environments and backdrops like <Environment>, <Stars> and <Sky> would make more sense in staging so I have moved those.
With effects and shaders I agree it's tricky to think of a succinct category name. How about visual effects?
I think we are getting there! The number of categories and things in each category feels quite nicely balanced.

I created a google doc so it's easy to discuss and quickly move things around without pushing lots of changes:
https://docs.google.com/document/d/1PPylp7XjcJS9HElB5jT7r-kPhuxTVcjCbzNi8OSQwZw/edit?usp=sharing
Staging is quite a broad term that can refer to the set dressing but also to how actors are positioned and move around the stage. So I was thinking that content would refer to the objects in the scene and staging would refer to how those things are lit and positioned, with things like <Float> , <Billboard> etc positioning the objects. I think you are right that environments and backdrops like <Environment>, <Stars> and <Sky> would make more sense in staging so I have moved those.
With effects and shaders I agree it's tricky to think of a succinct category name. How about visual effects?
I think we are getting there! The number of categories and number of things in each category feels quite nicely balanced.

Thanks! I took a stab at it as well. I think the proposal in that doc was a bit vague and arbitrary about where some of the things were put in. I really don't like the"Visual effects" category because it's going to be a generic toss-in-everything-that-looks-good bag.
It might be my personal preference, but I'd look for familiar naming so I proposed sorting into categories like: Objects, Geometries, Materials, Lighting & Shadows.
Not everything fits perfectly (yet?)

@jerzakm thanks for your input! Naming categories more specifically (geometries, materials) does make things clearer. There will be awkward things like meshline that fall between categories but I suppose that's what misc is for?

That being said, we should decide on something in the next day or two and just merge it. Tweaking docs is not a breaking change any ANYTHING is better than what we have now.

Agree with this! @ixxie do you want to have a look at our suggestions in the google doc and make a final decision on a category names and I'll make the changes?

I'm very happy with @edsunman's categories and names as they are now in the doc. The problem with being more specific @jerzakm, is that we end up with many categories. This both extends the length of the sidebar, and makes things harder to find since you need to parse more headers. As Ed mentions earlier, its a balance: you want Goldilocks categories which are neither too big, nor too small.

@ixxie I've pushed those changes so the order matches now the google doc.


micro_ros_arduino will stop after 30~40 minutes.

Hi! Thank you for the great project! I am working on a project using ROS2 robot. I am trying to connect with the micro ROS agent from my Teensy board. It was successful. However, teensy will stop after 30~40 minutes.
Does it occur in your environment?

Hardware description:
Tested board: Teensy 4.0 and 4.1

Installation type:
micro-ros-Agent: I was followed micro-ros instruction
micro-ros-Arduino: I was test each way arduino IDE and platformio.(In platformio used rebuilded library)

system information (please complete the following information):
OS: Ubuntu 20.04
ROS 2 Foxy

Version
micro_ros_setup: foxy (#275 Commits on Feb 11, 2021)
micro_ros_arduino: foxy (#99~#154 every version tested)
Micro-DDS-XRCE: foxy (#219 Commits on Feb 15, 2021)

Steps to reproduce the issue

Upload the script using Arduino IDE or Platformio.(script: micro-ros_recconectioin_example and micro-ros_publisher)

Connect micro-ros agent
# Terminal 1
ros2 run micro_ros_agent micro_ros_agent serial --dev [boardID]] -v6

Output messages from a topic
# Terminal 2
ros2 topic echo /int32_publisher_rclc

publisher will stop

micro-ros-agent and topic echo terminal will stop update

# Terminal 1
0020: 00 00 00 00
[1614855668.344434] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0x75078075, len: 16, data:
0000: 80 00 00 00 02 01 08 00 00 09 FF FD 02 00 00 00
[1614855668.344590] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0x75078075, len: 36, data:
0000: 80 00 00 00 06 01 1C 00 00 09 FF FD 00 00 01 0D 58 52 43 45 01 00 01 0F 00 01 0D 00 01 00 00 00
0020: 00 00 00 00 #<- this is the last message.

# Terminal 2
data: 1
~~~~~~
data: 2106
---
data: 2107
---
data: 2108
---       #<- this is the last message

Thanks in advance!

I'm going to do some tests and I'll come back

I have able to reproduce it, I'm trying to find a bug in the timing of the transport, I'll come back with a solution.
Meanwhile, @anaelle-sw have you found something related to this? Are you running your use-cases for more than 40 minutes?

Is possible for you @Tatsuya-2  to test this branch: https://github.com/micro-ROS/micro_ros_arduino/pull/158. Just checkout and run the example, is not necessary to rebuild the micro-ROS precompiled library.
I have had it running for more than an hour without crashing:

One of my guesses are that micros() functions rolls over every ~70 minutes, your publisherr have counted up to 2108 seconds which is 35 minutes (70/2), maybe there was some kind of problem in this timeout handling.
In the new branch, I have set the Teensy transport to the default Arduino ones which use Arduino API for timing out the reading operation: check here.
Let me know if this branch solves this issue and in that case, we will merge it.

Hello,
On the robot I am using for development, the Teensy application itself has always run well without stopping.
Some weeks ago, I remember having let ROS, micro-ROS, and the Teensy run for a quite long time (at least a complete day), and I didn't observe this kind of problem.
But in the last weeks, I did not let ROS (and micro-ROS-agent) run for more than 20 to 30 minutes on this robot (as we have problem with the agent regularly crashing). The Teensy application is still running on long period, but it is not connected to an agent, so I have no recent feedback about this.

Thank you very much for your support! @pablogs9
I will test it and report the result.

Is possible for you @Tatsuya-2 to test this branch: #158. Just checkout and run the example, is not necessary to rebuild the micro-ROS precompiled library.
I have had it running for more than an hour without crashing:

One of my guesses are that micros() functions rolls over every ~70 minutes, your publisherr have counted up to 2108 seconds which is 35 minutes (70/2), maybe there was some kind of problem in this timeout handling.
In the new branch, I have set the Teensy transport to the default Arduino ones which use Arduino API for timing out the reading operation: check here.
Let me know if this branch solves this issue and in that case, we will merge it.

@pablogs9, It works!
I tested recconection_example using feature/fix_teensy_transports branch and still working more than 1 hours.
I really appreciate for your help! :)

I still have errors when it is running for more than 70 minutes. I'm going to merge the patch and I will comment here if I found something else.

I still have errors when it is running for more than 70 minutes. I'm going to merge the patch and I will comment here if I found something else.

I had the same problem too. I think a micros() overflow is the problem as you said.

I'm starting to think about clock_gettime...
I'm going to check if it is possible to make a rollover safe function.

Please check and try this fix: https://github.com/micro-ROS/micro_ros_arduino/pull/159
My testing was done here<EMAIL_ADDRESS>

Please check and try this fix: #159
My testing was done here<EMAIL_ADDRESS>
Ok, I will test it from now until 3/9 and report the result at 3/9.

Interim report for you @pablogs9 : recconection_example is still working using #159 for 2 hours. It looks fixed this issue. :)
I will continue to test until 3/9 morning.

@pablogs9
Teensy 4.0 recconection_example is still publishing the topic.
It looks completely solved. :)
data: 305348   # <-- Approximately 85 hours
---

Thank you for your cooperation!

@pablogs9
Teensy4.0 recconection_example is still publishing the topic.
It looks completely solved this issue. :)
Thank you for your support!

I do not know how you have solved this issue. I was testing my arduino board nucleo64. I could be able to receive data until 71 minutes. Also I have tested my board two time and both 71 minutes.
How did you solve this issue?

Which board is  arduino board nucleo64`? Is supported?

The main change that solved this issue was: https://github.com/micro-ROS/micro_ros_arduino/pull/159/files

Hello - I'm using an OpenCR 1.0 and believe that I have the same issue.
I have the OpenCR publishing a topic at 10Hz using a timer, which runs fine for just under 35.79 min, at which point, the host appears to freeze.
I know that the OpenCR is continuing to run, as the main loop has a toggle for one of the onboard LEDs, which continues after the freeze (i.e. OpenCR not locked up), however a separate toggle that runs in a 1Hz timer callback stops, so that LED remains in whatever state it was at the time of the hang (and no messages from that timer are displayed on the terminal.
I accidentally left the OpenCR and terminal running while in a meeting, and noticed later that my clock display (on a browser-based UI) said 178.96 min (instead of 35.79), then observed that the terminal began to scroll data from the OpenCR sometime after 215 minutes (based on the OpenCR onboard clock).  Watched it over the next few hours, and found that it runs for 35.79 normally, freezes for that long, picks up again and runs normally for another 35.79 and repeats.  This continued for about 6.5 hrs when I finally shut things down.
Feels like a rollover or signed/unsigned  mismatch or similar issue, given that it works normally for exactly half the time, then hangs (as if an invalid time value is being handled) for the other half of the time, then repeats.
I naively tried to adapt the Teensy modifications in the opencr_transports and related files as appears in https://github.com/micro-ROS/micro_ros_arduino/pull/159/files, suspect I got something wrong.
Could I ask for an update to the OpenCR code as well?
Thanks!
Mark

Hello @ottensmeyer I have created this PR: https://github.com/micro-ROS/micro_ros_arduino/pull/357
Can you check out this PR's branch and test if it works?
What I have done is to force OpenCR to use default_transport.cpp that uses the Arduino API, so it should handle correctly the time. Let me know if this solves your issue and if not please open another issue mentioning this so we can track this problem

Hi Pablo,
confirming your suggestion - it looks like I should delete:
src/arduino_transports.c
and
src/opencr_transports.c.in
and replace
src/default_transport.cpp
with the new version that you put together - is that correct?
Will get back to you re: other issue or not later this week - when
something takes 36 min to verify that something's broken and over an hour
to find out if it's made any difference, testing is taking a while.  Looks
like I also have to work through impact of upgrading to 1.0.0/most recent
updates.  I tried 1.0.0 the other day, which caused other things to
"break", so I'm going back again to get to where I was first.
thanks
Mark
On Tue, Jul 6, 2021 at 1:43 AM Pablo Garrido @.***>
wrote:

Hello @ottensmeyer https://github.com/ottensmeyer I have created this
PR: #357 https://github.com/micro-ROS/micro_ros_arduino/pull/357
Can you check out this PR's branch and test if it works?
What I have done is to force OpenCR to use default_transport.cpp that
uses the Arduino API, so it should handle correctly the time. Let me know
if this solves your issue and if not please open another issue
mentioning this so we can track this problem
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/micro-ROS/micro_ros_arduino/issues/155#issuecomment-874476946,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEPGIF56F3B335NBM3I33LTWKJZTANCNFSM4YTDWULA
.

Ok, I'm merging the PR #357


[Event Request] Codeunit 90 Purch.-Post / CheckAndUpdate: OnBeforeCheckCorrectDocNoMandatory

We need a new event function in Codeunit 90 "Purch.-Post", named OnBeforeCheckCorrectDocNoMandatory:
LOCAL [IntegrationEvent] OnBeforeCheckCorrectDocNoMandatory(VAR PurchaseHeader : Record "Purchase Header";VAR IsHandled : Boolean)
called from:
LOCAL CheckAndUpdate(VAR PurchHeader : Record "Purchase Header")
....
....
....
IF Invoice THEN
CheckFAPostingPossibility(PurchHeader);
CheckPostRestrictions(PurchHeader);
CheckICDocumentDuplicatePosting(PurchHeader);
//INIT-INSERT
OnBeforeCheckCorrectDocNoMandatory(PurchHeader,IsHandled);
IF NOT IsHandled THEN
//END-INSERT
IF IsCreditDocType AND Invoice THEN BEGIN
IF PurchSetup."Correct. Doc. No. Mandatory" THEN
TESTFIELD("Corrected Invoice No.")
ELSE BEGIN
....
....
....
Also let me request a change in the code of the same procedure:
....
....
....
IF PurchSetup."Correct. Doc. No.
To apply this code the next local var will be necessary:
Name	                        DataType	    Subtype	Length
ConfirmManagement	Codeunit	    Confirm Management

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


license issues

as flagged in https://github.com/rhasspy/piper/issues/93
to comply with espeak-ng license this repo needs to be relicensed as GPL, unless the phonemizer can be replaced by a non-GPL alternative
since this plugin can NOT be used without espeak-ng it should be considered a derivative work
it should be added to the list of licensing exceptions in our docs https://openvoiceos.github.io/ovos-technical-manual/license/

First glance, all I can say is, yup. Will take a closer look after late lunch, marked as unread

For the benefit of OVOS folks who don’t click through to the issue at Piper:
Piper will be removing the hard GPL dep in an upcoming release, at which point this issue can be closed.

For the benefit of OVOS folks who don’t click through to the issue at Piper:
Piper will be removing the hard GPL dep in an upcoming release, at which point this issue can be closed.

we do not need to wait for that, we can replace espeak phonemizer, probably makes sense to even use the exisintg ovos G2P plugin framework  (and update corresponding espeak plugin for feature parity)

Replacing the phonemizer will likely require retraining the voices, unfortunately. Is all of your g2p stuff in Python? I've considered rewriting gruut in C++ or Rust one of these days, but it would require porting num2words as well.

The subject of rewriting components in machine code comes up a lot, but so far it’s mostly talk. There’s a port of your listener in C++. Otherwise, it’s a massive stack of Python.
Personally, I’ve been growing fonder and fonder of Rust. Evangelizing will be the hard part. I’d RIIR tomorrow.

for context, here are the docs for our G2P plugins
https://openvoiceos.github.io/ovos-technical-manual/g2p_plugins/

as for num2words, have you considered lingua_franca? i think it handles many more number variations in spoken format

I have, but it's Python only like num2words.

Piper will be removing the hard GPL dep in an upcoming release, at which point this issue can be closed.

as a small follow up to this, the GPL code now lives in it's own repo for piper-phonemize
others raised similar concerns there https://github.com/rhasspy/piper-phonemize/issues/17
it's still a hard dependency of this plugin since it won't work without it, so i'm not sure this issue can be closed yet


Multiple schemes

Hi,
Is there a way to use the new way of taking screenshots using multiple schemes?
If i create a scheme only 1 configuration can be tested / producing screenshots.
Regards,
Bas

I am sorry spent whole day trying to find a solution :) The solution is to auto create a test scheme.
See here my configuration:


FTA handler changes

Made changes to FTA, Com, ARP, Protocol handler such that the keys are deleted from the virtual mount if present before copying rest of the keys over to the actual registry.

}

this can just call UnloadMountedHive. make sure the success case doesn't try to unload it twice (can set m_hiveFileNameExists = false after unloading it successfully)

Refers to: preview/MsixCore/MsixCoreInstaller/RegistryDevirtualizer.cpp:598 in e635302. [](commit_id = e635302e8ebb02584950a742545c4cfd39248e32, deletion_comment = False)

Modified UnloadMountedHive method to set m_hiveFileNameExists to false in case the hive was successfully unloaded(ERROR_SUCCESS) so that it won't be unloaded a second time despite which ever case it gets called from for the first time.


[ONNX] Add null node.

Optional inputs in ONNX standard are represented by empty string.
We need a placeholder to keep information which inputs are not provided.

I think you also need a function like virtual bool Node::is_null_node() const to be able to check whether we are dealing with this class.

@diyessi what do you think about it?

I like that better than checking for the class type.

@diyessi
Please add a comment to null_node.hpp explaining what this is for.

Comment has been added.


Error when running the code

Hi,
I try to deduplicate my plain text file, but it shows some errors. I first run
python scripts/make_suffix_array.py c4-train.00000-of-01024.txt

The output is
./target/debug/dedup_dataset make-part --data-file c4-train.00000-of-01024.txt --start-byte 0 --end-byte 114700294
./target/debug/dedup_dataset make-part --data-file c4-train.00000-of-01024.txt --start-byte 114600294 --end-byte 229300588
./target/debug/dedup_dataset make-part --data-file c4-train.00000-of-01024.txt --start-byte 229200588 --end-byte 343900882
./target/debug/dedup_dataset make-part --data-file c4-train.00000-of-01024.txt --start-byte 343800882 --end-byte 458401177
Waiting for jobs to finish
Checking all wrote correctly
FACT 4.0
FACT 4.0
FACT 4.0
FACT 4.0
Rerunning 0 jobs because they failed.
Merging suffix trees
./target/debug/dedup_dataset merge --output-file tmp/out.table.bin --suffix-path c4-train.00000-of-01024.txt.part.0-114700294 --suffix-path c4-train.00000-of-01024.txt.part.114600294-229300588 --suffix-path c4-train.00000-of-01024.txt.part.229200588-343900882 --suffix-path c4-train.00000-of-01024.txt.part.343800882-458401177 --num-threads 256
thread 'thread '<unnamed><unnamed>' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rscalled `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:', 875src/main.rs::125222
:note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
77
thread 'thread 'thread '<unnamed><unnamed><unnamed>' panicked at '' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', ', ', src/main.rssrc/main.rssrc/main.rs:::222875875:::77125125

thread 'thread '<unnamed><unnamed>thread 'thread '' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }<unnamed><unnamed>called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', ' panicked at '' panicked at '', src/main.rscalled `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }src/main.rs:', ', src/main.rssrc/main.rs:222::875222:222::77:1257777

thread 'thread '<unnamed><unnamed>' panicked at '' panicked at 'thread 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }<unnamed>', ', src/main.rs' panicked at 'src/main.rs:called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:875', 875:src/main.rs:125:thread '125
222<unnamed>
:' panicked at '77called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }
', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread 'thread '<unnamed><unnamed>' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', thread '', src/main.rs<unnamed>src/main.rs:' panicked at ':222thread 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }875:<unnamed>', :77' panicked at 'src/main.rs125thread '
called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:
<unnamed>', 222thread '' panicked at 'src/main.rs:<unnamed>called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:77' panicked at '', 222
called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }src/main.rs:', :src/main.rs77222:
:22277:
77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }thread '', <unnamed>src/main.rs' panicked at ':called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }875', :src/main.rs125:
875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread 'thread 'thread '<unnamed><unnamed><unnamed>' panicked at '' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', ', ', src/main.rssrc/main.rssrc/main.rs:::222222222:::777777

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread 'thread 'thread 'thread 'thread '<unnamed><unnamed><unnamed>' panicked at '<unnamed><unnamed>' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', ', src/main.rs', ', src/main.rssrc/main.rs:src/main.rssrc/main.rs::222::222222:222222::77::7777
7777

thread 'thread 'thread 'thread 'thread 'thread 'thread '<unnamed><unnamed><unnamed><unnamed><unnamed><unnamed><unnamed>' panicked at '' panicked at '' panicked at '' panicked at '' panicked at '' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', ', ', ', ', ', ', src/main.rssrc/main.rssrc/main.rssrc/main.rssrc/main.rssrc/main.rssrc/main.rs:::::::222222222222222222222:::::::77777777777777

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread 'thread '<unnamed><unnamed>' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }thread 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }thread 'thread 'thread '', <unnamed><unnamed>', <unnamed><unnamed>src/main.rsthread '' panicked at '<unnamed>' panicked at 'src/main.rsthread 'thread 'thread 'thread 'thread '' panicked at '' panicked at ':called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:<unnamed><unnamed><unnamed><unnamed><unnamed>called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }222', called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', 222' panicked at '' panicked at '' panicked at '' panicked at '' panicked at '', ', :src/main.rs', src/main.rs:called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }src/main.rssrc/main.rs77:src/main.rs:77', ', ', ', ', ::
222:thread '222thread '
src/main.rssrc/main.rssrc/main.rssrc/main.rssrc/main.rs875222:22277:<unnamed>:thread '<unnamed>::::thread '::thread ':
77' panicked at '77<unnamed>' panicked at '875222222222<unnamed>222125<unnamed>77
called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }
' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }::::' panicked at ':
' panicked at '
', called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', 125777777called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }77called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }src/main.rs', src/main.rs

',
', :src/main.rs:src/main.rssrc/main.rs222:222::222:222:222:77::7777
7777

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }thread '', <unnamed>src/main.rs' panicked at 'thread ':called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }<unnamed>875', :' panicked at 'src/main.rsthread '125called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:<unnamed>
', 222' panicked at ':src/main.rscalled `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }77:',
222src/main.rs::77222
:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', thread 'src/main.rs<unnamed>:' panicked at '875called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:', 125src/main.rs
:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread 'thread '<unnamed><unnamed>' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', ', src/main.rssrc/main.rs::222875::77125

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', thread 'src/main.rs<unnamed>:' panicked at '222called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:', 77src/main.rs
:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }thread '', <unnamed>src/main.rs' panicked at ':called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }222', :src/main.rs77:
222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:875:thread '125<unnamed>
' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', thread 'src/main.rs<unnamed>:' panicked at '222called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:', 77src/main.rs
:875:125
thread 'thread '<unnamed><unnamed>' panicked at '' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', ', src/main.rssrc/main.rsthread '::<unnamed>thread '222875' panicked at '<unnamed>::called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }' panicked at '77125', called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }

src/main.rs', :src/main.rs875::222125:
77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'thread 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }<unnamed>', ' panicked at 'src/main.rscalled `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }:', 222src/main.rs::77875
:125
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', src/main.rs:222:77
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Any { .. }', /home/yiming/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-0.3.2/src/scoped.rs:34:43
Now merging individual tables
Cleaning up

Yet, it successfully create the suffix array file
c4-train.00000-of-01024.txt.part.0-114700294
c4-train.00000-of-01024.txt.part.0-114700294.table.bin
c4-train.00000-of-01024.txt.part.114600294-229300588
c4-train.00000-of-01024.txt.part.114600294-229300588.table.bin
c4-train.00000-of-01024.txt.part.229200588-343900882
c4-train.00000-of-01024.txt.part.229200588-343900882.table.bin
c4-train.00000-of-01024.txt.part.343800882-458401177
c4-train.00000-of-01024.txt.part.343800882-458401177.table.bin
c4-train.00000-of-01024.txt.table.bin

Then, I run
cargo run self-similar --data-file c4-train.00000-of-01024.txt --length-threshold 15 --cache-dir cache --num-threads 128

It gives me below error:
    Finished dev [optimized + debuginfo] target(s) in 5.69s
     Running `target/debug/dedup_dataset self-similar --data-file c4-train.00000-of-01024.txt --length-threshold 15 --cache-dir cache --num-threads 128`
Start load!
thread 'main' panicked at 'assertion failed: metadata.len() % (text.len() as u64) == 0', src/main.rs:479:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

May I ask how to fix this? Thank you!
Yiming

From the README:
(When running on larger files, if you get an error that you have too many open files, that's because this script opens lots of files. You should run ulimit -Sn 1000000 to "fix" the error. You might want to do this preemptively before hitting this crash after hour ten of the job.)

Whoops posted too soon: But what I'll try and do is catch this crash earlier and let the user know something has gone wrong. Leaving open until I improve the debug message.

Thank you for the reply. I set the ulimit, and now I can generate the table without error.
However, when I run
cargo run self-similar --data-file c4-train.00000-of-01024.txt --length-threshold 15 --cache-dir cache --num-threads 128

It still gives the same error:
    Finished dev [optimized + debuginfo] target(s) in 11.33s
     Running `target/debug/dedup_dataset self-similar --data-file c4-train.00000-of-01024.txt --length-threshold 15 --cache-dir cache --num-threads 128`
Start load!
thread 'main' panicked at 'assertion failed: metadata.len() % (text.len() as u64) == 0', src/main.rs:479:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I think this is same as another issue #11. May I ask how to fix this?
Yiming

If you pull and re-run it should fix things, I hope. (But please let me know if not, you'll be the first person to test it out.) It was an issue where the tmp directory was not cleared out if a run crashed.

I've also pushed a commit that should now warn early if you hit a crash because the merge fails. Do let me know if something is wrong though so I can fix.

Hi,
Thank you for the reply.
I re-pull the code and run again, but it still gives me the same error. In addition, I make sure that the tmp directory is empty before I start.

Can you show me the output of each of
ls -l c4-train.00000-of-01024.txt
ls -l c4-train.00000-of-01024.txt.table.bin
xxd c4-train.00000-of-01024.txt.table.bin | head

ls -l c4-train.00000-of-01024.txt
-rw-rw-r-- 1 yiming yiming 455496336 Apr 16 00:42 c4-train.00000-of-01024.txt

ls -l c4-train.00000-of-01024.txt.table.bin
-rw-r--r-- 1 root root<PHONE_NUMBER> Apr 16 00:55 c4-train.00000-of-01024.txt.table.bin

xxd c4-train.00000-of-01024.txt.table.bin | head
00000000: 8f52 261b 3500 0000 6300 0000 9e00 0000  .R&.5...c.......
00000010: d100 0000 2d01 0000 5701 0000 8401 0000  ....-...W.......
00000020: b301 0000 ce01 0000 f101 0000 1902 0000  ................
00000030: 6702 0000 0903 0000 3e03 0000 b003 0000  g.......>.......
00000040: e003 0000 0b04 0000 3404 0000 9c04 0000  ........4.......
00000050: d604 0000 1805 0000 4205 0000 c805 0000  ........B.......
00000060: 0e06 0000 3206 0000 6e06 0000 ab06 0000  ....2...n.......
00000070: ec06 0000 9307 0000 a007 0000 3708 0000  ............7...
00000080: 5108 0000 7808 0000 b608 0000 e208 0000  Q...x...........
00000090: 2d09 0000 7b09 0000 cd09 0000 f909 0000  -...{...........

Huh. That's weird, there are 44 extra bytes in this table file somehow...
Could you try deleting the file and re-generating it? I'm curious if there was some transient error... I don't know what could have caused the file size error.
Maybe also you could try generating with something smaller like 16 threads, and maybe see if the problem only arrises with 128? I'm not sure right now what would cause this.

I modify
print("./target/debug/dedup_dataset merge --output-file %s --suffix-path %s --num-threads %d"%("tmp/out.table.bin", torun, mp.cpu_count()))
pipe = os.popen("./target/debug/dedup_dataset merge --output-file %s --suffix-path %s --num-threads %d"%("tmp/out.table.bin", torun, mp.cpu_count()))

to
print("./target/debug/dedup_dataset merge --output-file %s --suffix-path %s --num-threads %d"%("tmp/out.table.bin", torun, 16))
pipe = os.popen("./target/debug/dedup_dataset merge --output-file %s --suffix-path %s --num-threads %d"%("tmp/out.table.bin", torun, 16))

and regenerate the file. It's working now. Thank you for the reply.

Okay well that's weird... Let me figure out what's going on with the number of threads being so high. You were using 128 before?

The node has 256 cores, but I use slurm script to allocate 200 cores for the job. Yet, the log shows that it uses 256 threads.

Hm okay. I'll try to debug. Leaving open until I can figure out why this happeend.

I've just got a chance to try this on a machine with >100 GPUs and I still can't crash it.
Would you mind trying a fresh clone and seeing if you can reproduce the issue still?

Closing.


Using composer create-project --no-dev gives an error

Using the --no-dev option fails to install whoops because it's added to composer require-dev.

Fatal error: Uncaught Error: Class 'Whoops\Run' not found in D:\projects\zend-expressive-skeleton-test\vendor\zendframework\zend-servicemanager\src\Factory\InvokableFactory.php on line 32

Without the --no-dev option it installs all develop requirements the first time which is why it was added to the docs. These packages are required for development and unit testing.
The solutions I can think of is:

Finding a way to remove the dev-requirements in a very early stage.
Rewrite the tests somehow without depending on the packages.
Make composer restart / reload composer.json after the installer is done configuring, if this is possible.
Use a second composer.json for development and travis? I don't know if this is even possible.

@xtreamwayz I found the same after testing the RC6 release (really, really wish it were simpler to use local repos with the create-project command to track this sort of thing down!). I figured you'd notice the issues and, hopefully, reach out to assist. :smile:
After I get the release announcement written and posted, and then take care of a few lingering zend-servicemanager issues, I'll be available to brainstorm and test some ideas. Do you have any pointers on how we might try out one of the first three ideas?

To test locally on windows I use a cmd and json file. I'm sure you can use the same for linux as well:
test-expressive-installer.cmd
@ECHO OFF

CALL rm -rf zend-expressive-skeleton-test/
CALL composer create-project --no-dev zendframework/zend-expressive-skeleton zend-expressive-skeleton-test --stability="dev" --keep-vcs --prefer-dist --repository-url="test-expressive-installer.json"
CD zend-expressive-skeleton-test
php -S localhost:8000 -t public

test-expressive-installer.json
{
    "package": {
        "name": "zendframework/zend-expressive-skeleton",
        "version": "1.0.0",
        "source": {
            "url": "/projects/zend-expressive-skeleton/.git",
            "type": "git",
            "reference": "feature/awesome-new-feature-branch"
        }
    }
}

This is also the reason why there are so many small commits sometimes :)
I'm doing some research now and see if I can find something.

I didn't see this issue before, sorry
I made some comments at #63 related to this one

skip error handler prompt if --no-dev is given; or
The reason the --no-dev is added is because without it, composer installs all dev-requirements on first install. Those packages are there for unit testing and developing the installer.

move filp/whoops to the require entries.
This might be a fast work around. However I think it should belong to the the dev dependency. Right now only whoops, phpunit and php_codesniffer are in require-dev, but this might change in the future. Basically it means that you can't run tests before doing composer update if you go this way.

Moving filp/whoops to the require section is not a good idea; it's not something you want installed in production. I think we can likely find a better solution; it just won't be obvious.

Found it. Working on a solution now.


Delete <shoot-name>.monitoring secret from project namespace when purpose is switched to testing

How to categorize this issue?

/area monitoring
/kind bug
What happened:
When an existing Shoot with a purpose!=testing is switched to purpose=testing then the monitoring stack is deleted: https://github.com/gardener/gardener/blob/61e391ecb37013c8fa0637136bb03ba9f4ddac91/pkg/operation/botanist/monitoring.go#L452-L454
However, this does not delete the <shoot-name>.monitoring secret from the project namespace
This secret is originally synced to the project namespace in the DeploySeedGrafana function: https://github.com/gardener/gardener/blob/61e391ecb37013c8fa0637136bb03ba9f4ddac91/pkg/operation/botanist/monitoring.go#L519-L525
We would need similar cleanup code like here: https://github.com/gardener/gardener/blob/61e391ecb37013c8fa0637136bb03ba9f4ddac91/pkg/operation/botanist/kubeapiserver.go#L480-L483
What you expected to happen:
The <shoot-name>.monitoring secret should disappear if no monitoring stack is deployed anymore.
How to reproduce it (as minimally and precisely as possible):

Create a shoot with purpose=evaluation.
See the <shoot-name>.monitoring secret appear in the project namespace
Update the shoot to purpose=testing.
Still see the <shoot-name>.monitoring secret existing in the project namespace

/assign


Swagger definition file has no return types...

For operations "get all objects" I get in swagger definition file (which is correct to my understanding):
"operationId":"GetAllPromotions","consumes":[],"produces":["application/json","text/json","application/xml","text/xml"],"responses":{"200":{"description":"OK","schema":{"type":"array","items":{"$ref":"#/definitions/PromotionDto"}}}}}
but for "get individual objects" type of operations swagger definition file is missing return type:
"operationId":"GetPromotionById","consumes":[],"produces":["application/json","text/json","application/xml","text/xml"],"parameters":[{"name":"id","in":"path","required":true,"type":"integer","format":"int64"}],"responses":{"200":{"description":"OK"},"404":{"description":"NotFound"}}
And this is excerpt from REST API controller
        [SwaggerOperation("GetPromotionById")]
        [SwaggerResponse(HttpStatusCode.OK)]
        [SwaggerResponse(HttpStatusCode.NotFound)]
        public Promotion Get(long id)
        {
            using (var db = new PromotionRepository())
            {
                var promo = db.GetById(id);
                if (promo == null)
                    return new Promotion();
                else
                    return promo;
            }
        }

Any ideas on how to solve this?

Does anyone have any recommendations regarding this issue? Is the information from my side sufficient? Is this a bug, or is this just wrong usage of the tool from my side?
It's been more than a week without any reaction...