How to get the distance of the vehicle from the center of the lane and the angle between the vehicle orientation and the road tangent

Thanks for Carla team hard work.
I  want to train a reinforcement learning agent based on CARLA.
How to get the distance of the vehicle from the center of the lane and the angle between the vehicle orientation and the road tangent？

Hi @HubFire.
Using 0.9.1 you are able to get a carla.Waypoint (based on the OpenDrive) using:
waypoint = client.get_world().get_map().get_waypoint(carla.Location, project_to_road=True)

As you have specified project_to_road=True, this waypoint will be on the center of the (nearest) road.
A carla.Waypoint have a carla.Transform that have a carla.Location and a carla.Rotation.
You can use this transform to get the distance of the center of the road and the orientation difference.
Cheers!

@marcgpuig I wanted to know more about the Waypoints. Here are my doubts on the matter:
(I do not expect direct answers to all of them, but it would be great if you could direct me to the relevant document / literature where I could find the answers).

How are waypoints represented in the Map.

When you call get_waypoint(Location):

Is the returned waypoint the nearest waypoint (from a list of waypoints in the map) to the Location?
or
Is it the transform of the nearest waypoint (point normal to the nearest road network) ?

How do I compute road curvature?

My present idea is to get the next few waypoints, fit a cubic spline over these points and get the curvature.
But, If you can direct me to literature regarding the road network and the waypoints used, I think there might be a better way to go about it.

When we call map.generate_waypoints(distance):

Why aren't the waypoints generated near the vicinity of the vehicle / spectator. How to make that happen?

When we call waypoint.next(distance):

Is the new waypoint precisely that euclidean distance away from the present point or is it the distance along the route.

Hi @marcgpuig I was wondering if there was any news on how to access the road geometry through the Python API. I would also need to compute the road curvature, maybe you can give me a hint on that? Thanks!

@YashBansod Did you finally get through with the calculating the curvature ? If yes then maybe you could pass on the knowledge :)

I use this approximation method.
curvature = distance_degree / distance_route.
"""python
def calcu_c(host_waypoint, route_distance, min_angel_dif):
previous_waypoint = host_waypoint.previous(route_distance)[0]
next_waypoint = host_waypoint.next(route_distance)[0]
total_route_distance = 2*route_distance
angel_dif = next_waypoint.transform.rotation.yaw-previous_waypoint.transform.rotation.yaw
# if angel_dif < min_angel_dif:
#     c_0 = 0
# else:
c_0 = radians(angel_dif) / total_route_distance
return c_0
"""

I use this approximation method.
        def calcu_c_1(host_waypoint, route_distance):
            previous_waypoint = host_waypoint.previous(route_distance)[0]
            next_waypoint = host_waypoint.next(route_distance)[0]
            _transform = next_waypoint.transform
            _location, _rotation  = _transform.location, _transform.rotation
            x1, y1 = _location.x, _location.y
            yaw1 = _rotation.yaw

            _transform = previous_waypoint.transform
            _location, _rotation  = _transform.location, _transform.rotation
            x2, y2 = _location.x, _location.y
            yaw2 = _rotation.yaw

            c = 2*sin(radians((yaw1-yaw2)/2)) / sqrt((x1-x2)**2 + (y1-y2)**2)
            return c

std and mean with different route_distance.

Hello! I'm having some problems implementing this method to calculate the road curvature. What do you mean with 'route_distance'? Has someone been able to implement this method or another similar one?
Thanks to everyone!

Hello! I'm having some problems implementing this method to calculate the road curvature. What do you mean with 'route_distance'? Has someone been able to implement this method or another similar one?

That waypoint api requires a distance argument to specify how far the previous or next waypoint you want. Based on the script @kannidekan provided, it is used to get the desired waypoint in "route_distance" ahead or behind.


Fixed the issue reported in #111

The fix was made adding a verification to the set parameters of the build function, so if it detects a "select" as parameter, it adds the lost parenthesis.

This fix is no longer needed now that we have fromQuery(), see http://hiddentao.github.io/squel/api.html#insert_fromquery


Build error

Launching lib\main.dart on Android SDK built for x86 in debug mode...

FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':webview_cookie_manager:generateDebugRFile'.
> Could not resolve all files for configuration ':webview_cookie_manager:debugCompileClasspath'.
   > Could not resolve io.flutter:flutter_embedding_debug:1.0.0-d1bc06f032f9d6c148ea6b96b48261d6f545004f.
     Required by:
         project :webview_cookie_manager
      > Cannot find a version of 'io.flutter:flutter_embedding_debug' that satisfies the version constraints:
           Dependency path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:1.0.0-d1bc06f032f9d6c148ea6b96b48261d6f545004f'
           Dependency path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1'
           Constraint path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:{strictly 1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1}' because of the following reason: debugRuntimeClasspath uses version 1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1

   > Could not resolve io.flutter:flutter_embedding_debug:1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1.
     Required by:
         project :webview_cookie_manager
      > Cannot find a version of 'io.flutter:flutter_embedding_debug' that satisfies the version constraints:
           Dependency path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:1.0.0-d1bc06f032f9d6c148ea6b96b48261d6f545004f'
           Dependency path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1'
           Constraint path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:{strictly 1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1}' because of the following reason: debugRuntimeClasspath uses version 1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1

   > Could not resolve io.flutter:flutter_embedding_debug:{strictly 1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1}.
     Required by:
         project :webview_cookie_manager
      > Cannot find a version of 'io.flutter:flutter_embedding_debug' that satisfies the version constraints:
           Dependency path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:1.0.0-d1bc06f032f9d6c148ea6b96b48261d6f545004f'
           Dependency path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1'
           Constraint path 'io.flutter.plugins.webview_cookie_manager:webview_cookie_manager:1.0' --> 'io.flutter:flutter_embedding_debug:{strictly 1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1}' because of the following reason: debugRuntimeClasspath uses version 1.0.0-a9d88a4d182bdae23e3a4989abfb7ea25954aad1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13s
Exception: Gradle task assembleDebug failed with exit code 1
Exited (sigterm)

Tried adding,
maven { url "https://storage.googleapis.com/download.flutter.io"}
to build.gradle file, also tried changing dns, but didn't work.

Please, feel free to open a PR

I have the exact same problem. Do you have any hints on how to solve this?

I found a solution for this issue: Update Flutter to Version 2.2.3

Same issue here. Just upgraded to Flutter version 2.5.1 then got this error

Same issue here. Just upgraded to Flutter version 2.5.1 then got this error.. Switched to Beta / Dev and fixed

Thank you so much for this suggestion - I wouldn't have thought that upgrading to an even higher version would work!
Flutter (Channel beta, 2.6.0-5.2.pre) works :)

Same issue here. Just upgraded to Flutter version 2.5.1 then got this error.

fixed with flutter channel dev > flutter upgrade

Please check in the new version.


Fixing NullReferenceException when accessing certain response types in a TrainAnnouncement

Hi, I started using your library recently but  the strings returned when trying to access ProductInformation, Deviation, Booking, OtherInformation, Service or TrainComposition parts of a TrainAnnouncement response were either empty or just null. I seem to have tracked down the issue to the fact that none of the public attributes for those classes had implemented getters and setters. I simply mapped all those attributes to their corresponding internal fields as was already done on other classes in a TrainAnnouncement. This seems to have fixed the issue and the results are now being accessed fine.
I also added .NET 7 as a target framework so i could actually run the library in my own code. There seems to be no compatibility issues compiling for .NET 7, but i haven't tested this extensively at all. Great work on the library, made my life a lot easier and I hope this little fix helps, even though this project hasn't been updated for a while.

Thank you for the kind words!
I have no idea how I could've possibly missed this, good catch!


Fix for #294 - reverse search crash - the tests still pass, but library doesn't crash

I commented out the assertion which was causing the problem reported in #294, and added the simple program which was causing the crash.

The assert should not be removed.
Would you mind testing PR #296 ?
And close this PR.
Thanks.


Larger simulations are not deterministic

To Reproduce
MGPUSim version of commit ID:
40c4cd4
Command that recreates the problem
./fir -length=65536 -timing

Current behavior
The estimated execution time differs from execution to execution
Expected behavior
The estimated execution time should be the same.

I want to know if this bug has been fixed. I found that the results of parallel engine are also different from those of serial engine.

The problem is still there.
If parallel simulation is used, the simulation will be underministic for sure. The scope of making the simulation deterministic only applies to single-kernel serial simulation.
Also, for parallel simulations, how different are they from serial simulations?

The problem is still there.
If parallel simulation is used, the simulation will be underministic for sure. The scope of making the simulation deterministic only applies to single-kernel serial simulation.
Also, for parallel simulations, how different are they from serial simulations?

I wonder what the possible cause of this problem is, Golang or mgpusim itself. If I know the possible reason, I may be able to try fixing this bug myself.

I can use the serial simulation currently. Thanks for your reply.

Prof.Sun, I think I may fix the bug about larger simulations are not deterministic.
Firstly, I record the scheduling and handling order of events and find that the access order of the device port of an endpoint is not deterministic (akita/noc/networking/switching/endpoint.go:sendFlitOut(now)), which causes that the Tick() function of different components connected to the endpoint may be executed randomly.

As the figures show, the left figure executes the RDMA firstly but the right figure executes the commandprocessor firstly. This is because when the timing platform plugin the device onto the endpoint (PlugInDevice(pcieSwitchID, gpu.Domain.Ports())), the result of gpu.Domain.Ports() is not deterministic.
So, I modify the codes of Port() and the bug looks like be fixed.

@MaxKev1n Looks great! Can you start a pull request, and I can look deeper into it?

BTW, there is a deterministic test script under test/deterministic. In the Python file, you can see that there is a line being commented out. You can reinclude that line and see if the problem is solved. If not solved, at least determinicity can last until what problem size.

In your deterministic test script, I find that running fir with single GPU is not able to reproduce the problem, so I run fir with 4 GPUs and reproduce the problem successfully. My code can eliminate the majority of deterministic problem except a set of metrics called CPIStack, I think CPIStack may have other problem. And I found that there may be a small difference in the total time of fir, but I think this is acceptable.

@MaxKev1n Thanks for the PR. I am merging it.
However, I do not think this problem is fully resolved given the small difference. Being fully deterministic is more about debugging. When we find a bug, we want to rerun the program and the bug takes place at the exact same location. We will keep looking into the problem. I think we are close.


refactor(about): remove generic JSON Resume service

Splits the JsonResumeAdapter into PositionsService and EducationItemsService. So everything is more encapsulated per feature. Also this way the service reponsibility is to retrieve data, no matter the source. The fact it comes from a JSON Resume it's an implementation detail. Could come from another source.

Current dependencies on/for this PR:

main

PR #77   👈

This comment was auto-generated by Graphite.


Optional view controllers in Storyboards

When I'm trying to instantiate a view controller from storyboard it returns an optional value. Why don't you use force unwrapped values in R.generated (because you know for sure the type of view controller)?

A design goal of R.swift is to never have the generated code crash at runtime.
People have asked about force unwrapping before, mostly in the context of images. For images there are legitimate reasons why the constructor might return nil. In particular for on-demand resources or watchOS-only resources on iOS.
Although these reasons for nil do not apply to view controllers, there is at least one way for this method to return nil at runtime; If the R.generated.swift file was generated at the different time as the storyboard used in the app.
For my personal projects, I've setup R.swift as a custom build step in Xcode (as described in the readme). So when I'm using R.swift, I'm sure the generated file and the storyboards used do match. Therefor I'm fine with force unwrapping in my own app: R.storyboard.planner.plannerNavigationController()!
However R.swift can't assume all users will use a similar setup. If, for some reason, the generated code doesn't match the runtime environment, a force unwrap would crash. If it needs to crash, we'd much rather have the force unwrap be in the code written by the user, instead of in generated code (and getting the tool be blamed for the crash).

@mac-cain13 We may want to add this explanation to the readme somewhere.

Feels like a good thing for the FAQ.

I just came here 3 years after the question has been asked. @tomlokhorst thanks for a great explanation. Still very helpful!
What do you think about adding a parameter/flag for making the view controllers in R.generated force unwrapped? This could be useful for users that follow the installation process described in Readme and are sure the generated file and the storyboards do match.


How to avoid to be see as a robot?

I try to crawl a website but there is a captcha appearing after a certain amount of page crawled. So i open my browser and fill the captcha manually.
Is there a way to avoid to do that or at least increase the number of page i can crawler before get this message. cooldown? hwo to?

I suspect the website owner does not want you crawling their website, in which case you should ask to be whitelisted. If this is just simple rate limiting, you can either:
(a) reduce the number of requests you are making; or
(b) distribute your requests over a number of machines/IPs. There are third-party services like Luminati.io that solve this problem, although I have no experience with them.

(a) is it possible to limit to crawl only pageswith URLs starting by a specific word. I would like limit by host name and by path starting by something. It will reduce a lot the number of requests.
(b) is it compatible with supercrawler to have different instances on the same db?

i too have these same questions @brendonboshell  (poke poke) thanks for building this!

The only way to control which URLs get added to the queue is using the content handlers. The htmlLinkParser takes every link on the page and adds it to the queue. You can build your own content handler that applies the filters you need (hostname/path).
You can run multiple instances of Supercrawler using the same database (i.e. DbUrlList or RedisUrlList). In my own use case, I have ~15 servers at 20 crawls/second using a single Redis server. I recommend using Redis in any case.
Does this help @design2dev ?

it helps thnks! also how easy is it to ignore their robots.txt? @benchristel


[FEATURE] OpenFaas support

Is your feature request related to a problem? Please describe.
Deployment of microservices is a pain, even with a manged k8s cluster you have to extensively describe deployments using e.g. helm to integrate it into a CI/CD pipeline.
Describe the solution you'd like
Add an option to deploy microservices developed using go micro with https://www.openfaas.com/. This is especially powerful with the https://github.com/openfaas-incubator/kafka-connector connector and would simplify deployments by a lot.
Additional context
https://www.openfaas.com/blog/stateless-microservices/

No support for now


Layer Crossing Definitions Package Versioning and Architecture

Currently within the fluid framework we have several definition packages that we version and release independently. These packages are:

@fluidframework/common-definitions
@fluidframework/protocol-definitions
@fluidframework/core-interfaces
@fluidframework/driver-definitions
@fluidframework/container-definitions

These packages have a specific goal which is to provide a stable api for any layer crossing object which is agnostic to specific implementation versions. The specific layers of concern here are server, loader, and runtime. These layers are special as the fluid framework explicitly supports independent compilation and packaging of each layer with no shared implementations. The definition packages specify the contracts across these layers and allow objects to move across these layers at runtime.
The versions of these packages have been pinned as an interim solution to meeting the above goal. However, pinning the versions is not sustainable long term. This issue will lay out several proposals which should allow us to meet our goal for these packages while enabling forward progress of the definitions packages and their respective implementations.
Remove non-layer crossing contracts
Within the current definitions packages there are some non-layer crossing types. These types unnecessarily bloat our definitions packages and limit our agility in modifying the implementation tied to them. In general, these types should be moved to the packages containing their respective implementations and/or new packages within those layers. Additionally, there may be some types that are unused, and can be fully removed.
Export Definitions as Types only
Typescript support both package imports and exports as types only. This means that there will be no runtime impact from those import or exports. We should update our definition packages to export everything as type only. By exporting as types only we can truly leverage duck typing for back compat, without needed to consider runtime compatibility. Runtime compatibility is a problem we will solve at the implementation layer. Practically this means the removal of all Enum and functions from our definition packages
Re-export upstream definitions
This is more a nice to have, but i think it provides a number of benefits.

Make it easier for consumers to manage definition dependencies, as they should only need a single import per layer
Gives us some flexibility in moving types between layers in a way that minimizes breaks for downstream consumers.
Makes peerDependencies more supportable

Validation
We currently have tests that validate our code is back compat at runtime, however we don't do any compile time validation. Some ideas here are leverage peer dependencies, and validate multiple compiles, likely old version code with new dependencies. another potential option is to leverage some of the type testing we do to ensure our new types are back compat to some point in the past.

I think we'll need to solve #3264, and #2566 before we can do much here, as the current Module augmentation used by IFluidObject doesn't work correctly with re-exporting, and in general will create a lot of cross version pain. The biggest unknown here is how to deal with types that leverage keyof IFluidObject. I don't think there is a one size fits all approach to replacing that, but i'm exploring the usage patterns to see what we can do.

we should revisit the finding here after #8077 #8007 and #8006 are complete, especially around re-export and exporting types, as those may make our versioning story more sustainable.


Handle bad credentials for scm widget

Invalid credentials should be handled gracefully by the github collector and displayed on the dashboard. Currently the collector keeps polling with invalid credentials.

Was thinking about this when working on the gitlab collector, need some way to show the user that there is an error, just not sure how to do it with what's currently available.

@satishc1 @rob-miller-777  - Also we may need to halt collectors when there is an error. My opinion,  we have to set a flag(or a value) to mongo and check for collector status. Use the same flag to propagate as an error in UI, code should handle to catch exception and terminate the collectors when occurred.  Thoughts?

Here is my design suggestion:
Set a flag in collectorItem indicating a potential bad "url" (including bad credential). Set it to true when encounter non 200 response multiple times (configurable via properties). Unset only via UI. Also, the UI widget should indicate that the "url" is encountering trouble.

Potentially can set "enabled" flag on the collector to false too. Only to be modified from UI.

I don't know about halting the collector, at least with github/gitlab, you can get errors when the user misconfigures there repo, or using a private repo without providing credentials.  This would then stop all repos from being collected.
My suggestion:
Create a new collection for "Dashboard Errors", which can be populated by any of the collectors.  Then add something in the UI for displaying the dashboard errors to the user.  There are probably other cases where things fail silently that we should be displaying to the users.

While you are at it - fix one more bug too:
The SCM widget creates duplicate collector items. Can you fix that too :-)

@rob-miller-777 The collector should stop polling a configured scm (collectorItem) as soon as it encounters an error. We could relax this rule if we can correctly identify the type of error. For example a network error is not reason enough to stop polling, but bad credentials or bad url is reason enough.

That would be easy enough for Gitlab, as both bad credentials, and bad url would result in a 404.  Even with that, it would still be worthwhile to alert the user so they can fix the problem.

@rob-miller-777 Definitely. We can read the flag on the collectorItem and update the status of the widget. The widget currently displays the time when the last poll happened. @nireeshT is familiar with this piece. The status could be right next to it.

@satishc1 @rob-miller-777 @nireeshT
I have done some initial POC to capture collection error and display on the widget.
Check this branch: https://github.com/capitalone/Hygieia/tree/issue-1139
-Added "error" as a list of a new error class to collectorItem.
-Capturing collector's client exception and adding to the error list.
-Collector properties file will have a threshold value for number of errors. When this threshold is reached, collector stops collecting for that collector item.

In UI: widget load function will return the errors in Promise. widget.js will handle that and display a "red dot" next to widget heading. A tooltip will show error. Example UI is here:

What do you think?

@tabladrum Nice. 👍  That looks great. Is the errors erased when they re-save the widget? How about this one http://fontawesome.io/icon/ban/ ?

I like it. When the threshold is reached, can we also alert the user that it's no longer trying to collect that repo in addition to the error message?  And how would you get it started again?  The reason I ask is what if your entire instance of gitlab goes down, and it hits that threshold for everything, would be nice to just click a button to get it collecting again for each repo.

This is what I am contemplating...

For now only Github needs this check. As the collector items are user created via the UI. Others are fetched by the collectors.
Potentially others may run into the same issue - if job config changes etc. I have seen this in NexusIQ too.
Each collector will have full control over what error condition to capture to "lock" the collection for that specific collector item.
Will implement this for Github  for now and carry it forward one collector at a time. The UI and API wont change.
Next round will be to lock down the whole collector (in the same way) when there is a global connectivity issue (say Github is down) and create a new Collector management Admin page to manage the collectors and potentially collector items from that page.

Thoughts?

Sounds good to me.  As far as the collector management admin page, I have been thinking it would be nice to have something like this for managing things like Jenkins/Sonar servers, instead of relying on properties files which require the application to be restarted to take effect.

@rob-miller-777 exactly. When @satishc1 is done with his work on that page, I will add this.


[BUG] Invalid data found when processing input (FFmpeg post processing)

Checklist

[X ] I'm reporting a broken site support issue
[X ] I've verified that I'm running youtube-dl version 2020.05.08
[X ] I've checked that all provided URLs are alive and playable in a browser
[X ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[ X] I've searched the bugtracker for similar bug reports including closed ones
[ X] I've read bugs section in FAQ

Verbose log
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', '/root/cookies.txt', '--prefer-ffmpeg', '-f', 'best', '-ciw', '--write-info-json', '--write-thumbnail', '--all-subs', '--add-metadata', '--ignore-errors', '-o', '/mnt/transfer/%(uploader)s/%(title)s .%(ext)s', '-v', 'https://www.youtube.com/channel/UCXuqSBlHAE6Xw-yeJA0Tunw']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.05.08
[debug] Python version 3.6.9 (CPython) - Linux-4.15.0-101-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 4.2.2, ffprobe 4.2.2
[debug] Proxy map: {}

s' 'file:/mnt/transfer/Linus Tech Tips/Our server room ACTUALLY Caught Fire Explained .temp.mp4'
ERROR: file:/mnt/transfer/Linus Tech Tips/Our server room ACTUALLY Caught Fire Explained .mp4: Invalid data found when processing input
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/youtube_dl/YoutubeDL.py", line 2065, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/lib/python3.6/dist-packages/youtube_dl/postprocessor/ffmpeg.py", line 498, in run
    self.run_ffmpeg_multiple_files(in_filenames, temp_filename, options)
  File "/usr/local/lib/python3.6/dist-packages/youtube_dl/postprocessor/ffmpeg.py", line 235, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
youtube_dl.postprocessor.ffmpeg.FFmpegPostProcessorError: file:/mnt/transfer/Linus Tech Tips/Our server room ACTUALLY Caught Fire Explained .mp4: Invalid data found when processing input

Description
Looks like than the ffmpeg is not able to do post processing not sure (I think)

this is a duplicate of #25717

Fixed in 2021.06.06


cbor deserialization refactor

Adds a CborDeserializer[T] trait that's extended by singleton objects for each CborSerializable type we want to decode.

Replaces the typeName match { ... }  construct in the fromCMap method with a Map[String, CborDeserializer[CborSerializable]] which is used to look up the deserializer for a given type name.

Removes the CborSerializable trait from the ChainReference types, since they're internal to the transactor.

Renames the TypeSerialization object to CborSerialization, since we also have pojo serialization for some types.

@bigs - the last commit is probably relevant to the datastore work you're doing; I made the CBORType value in CborSerializable optional, and it's now represented with case objects that extend the CborTypeName trait.
Let me know if you think there's a better way to do this btw :)

@yusefnapora this is perfect, thanks!

:+1: for multihash references.


Missing change to Readme

I updated the sample JSON but forgot to also change the readme accordingly. Now it looked like you would get that new output with the 0.4 version... fixed..

:+1:

:+1:

:+1:

:+1:

:+1:


Internationalisation documentation issue 2.11rc1

Maybe docs need to improve slightly.
I have setup wagtail. According docs. International seems to work. I see in admin hen creating a page choice for language.  I also see in API the locale tag. So I think I set everything correct up.
However Its unclear how to translate a page. I don't see any button or any way to translate.
I do the following:

create homepage (english)
link that homepage in wagtail setting as root page
???

tried to copy the page
tried to make extra homepage in dutch
tried to create 2 wagtail configurations
tried to reconfigure, restart project etc ...

Nothing works to setup the translation. Nor the ?translation_of=... gives me translations
I'm missing some crucial information how to start and how the wagtail tree is copied over. I more ore less expect a button so I can jump from the english version to the dutch version.
Or some setting in base.py where you can say copy_tree=True ...
My request how to make docs a bit more clear? I must doing something wrong?
The docs say
_Translated homepages
When you set up a site in Wagtail, you select the site’s homepage in the ‘root page’ field and all requests to that site’s root URL will be routed to that page.
Multi-lingual sites have a separate homepage for each locale that exist as siblings in the page tree. Wagtail finds the other homepages by looking for translations of the site’s ‘root page’.
This means that to make a site available in another locale, you just need to translate and publish its homepage in that new locale.
If Wagtail can’t find a homepage that matches the user’s language, it will fall back to the page that is selected as the ‘root page’ on the site record, so you can use this field to specify the default language of your site.

Hi @onno-timmerman,
The internationalisation support built into Wagtail 2.11 only provides the infrastructure for creating and serving content in multiple languages - for managing translations between languages, you additionally need the wagtail-localize app. Will update the docs to make this clearer.

A follow up question for me would be how to link multiple languages to one site in site settings?
It would be nice if you can link one site to multiple roots (with each different language of course) in the site settings.


Investigate collecting data passively

Currently the application collects data only when the data collection service is active (either started manually or automatically when the device is moving). Android supports using passive location provider to listen for locations that other apps have requested. Investigate if it would be possible to create reports in the background by using the passive location provider

This could surely save battery.


End-to-End Experience Isn't Functional After Following Tutorial Steps

On: https://nftschool.dev/tutorial/end-to-end-experience/#getting-set-up
After following the tutorial, and ultimately launching the web page "connecting..." is displayed in the upper-right corner, instead of the NFTMinter:
This issue was reported to me by a developer trying to follow along using Windows 10. I attempted to follow along using Manjaro and encountered the exact same issue.
Following the steps outlined, I get the following outputs:
yarn chain:
[user@TheDesktop end-to-end]$ yarn chain
yarn run v1.22.10
$ yarn workspace @scaffold-eth/hardhat chain
$ hardhat node
✖ Help us improve Hardhat with anonymous crash reports & basic usage data? (Y/n) · true
Started HTTP and WebSocket JSON-RPC server at http://<IP_ADDRESS>:8545/

Accounts
========
Account #0: 0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266 (10000 ETH)
Private Key: 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80

Account #1: 0x70997970c51812dc3a010c7d01b50e0d17dc79c8 (10000 ETH)
Private Key: 0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d

Account #2: 0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc (10000 ETH)
Private Key: 0x5de4111afa1a4b94908f83103eb1f1706367c2e68ca870fc3fb9a804cdab365a

Account #3: 0x90f79bf6eb2c4f870365e785982e1f101e93b906 (10000 ETH)
Private Key: 0x7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6

Account #4: 0x15d34aaf54267db7d7c367839aaf71a00a2c6a65 (10000 ETH)
Private Key: 0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a

Account #5: 0x9965507d1a55bcc2695c58ba16fb37d819b0a4dc (10000 ETH)
Private Key: 0x8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba

Account #6: 0x976ea74026e726554db657fa54763abd0c3a0aa9 (10000 ETH)
Private Key: 0x92db14e403b83dfe3df233f83dfa3a0d7096f21ca9b0d6d6b8d88b2b4ec1564e

Account #7: 0x14dc79964da2c08b23698b3d3cc7ca32193d9955 (10000 ETH)
Private Key: 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356

Account #8: 0x23618e81e3f5cdf7f54c3d65f7fbc0abf5b21e8f (10000 ETH)
Private Key: 0xdbda1821b80551c9d65939329250298aa3472ba22feea921c0cf5d620ea67b97

Account #9: 0xa0ee7a142d267c1f36714e4a8f75612f20a79720 (10000 ETH)
Private Key: 0x2a871d0798f97d79848a013d4936a73bf4cc922c825d33c1cf7073dff6d409c6

Account #10: 0xbcd4042de499d14e55001ccbb24a551f3b954096 (10000 ETH)
Private Key: 0xf214f2b2cd398c806f84e317254e0f0b801d0643303237d97a22a48e01628897

Account #11: 0x71be63f3384f5fb98995898a86b02fb2426c5788 (10000 ETH)
Private Key: 0x701b615bbdfb9de65240bc28bd21bbc0d996645a3dd57e7b12bc2bdf6f192c82

Account #12: 0xfabb0ac9d68b0b445fb7357272ff202c5651694a (10000 ETH)
Private Key: 0xa267530f49f8280200edf313ee7af6b827f2a8bce2897751d06a843f644967b1

Account #13: 0x1cbd3b2770909d4e10f157cabc84c7264073c9ec (10000 ETH)
Private Key: 0x47c99abed3324a2707c28affff1267e45918ec8c3f20b8aa892e8b065d2942dd

Account #14: 0xdf3e18d64bc6a983f673ab319ccae4f1a57c7097 (10000 ETH)
Private Key: 0xc526ee95bf44d8fc405a158bb884d9d1238d99f0612e9f33d006bb0789009aaa

Account #15: 0xcd3b766ccdd6ae721141f452c550ca635964ce71 (10000 ETH)
Private Key: 0x8166f546bab6da521a8369cab06c5d2b9e46670292d85c875ee9ec20e84ffb61

Account #16: 0x2546bcd3c84621e976d8185a91a922ae77ecec30 (10000 ETH)
Private Key: 0xea6c44ac03bff858b476bba40716402b03e41b8e97e276d1baec7c37d42484a0

Account #17: 0xbda5747bfd65f08deb54cb465eb87d40e51b197e (10000 ETH)
Private Key: 0x689af8efa8c651a91ad287602527f3af2fe9f6501a7ac4b061667b5a93e037fd

Account #18: 0xdd2fd4581271e230360230f9337d5c0430bf44c0 (10000 ETH)
Private Key: 0xde9be858da4a475276426320d5e9262ecfc3ba460bfac56360bfa6c4c28b4ee0

Account #19: 0x8626f6940e2eb28930efb4cef49b2d1f2c9c1199 (10000 ETH)
Private Key: 0xdf57089febbacf7ba0bc227dafbffa9fc08a93fdc68e1e42411a14efcf23656e

web3_clientVersion
eth_chainId
eth_accounts
eth_chainId
eth_estimateGas
eth_gasPrice
eth_sendTransaction
  Contract deployment: <UnrecognizedContract>
  Contract address:    0x5fbdb2315678afecb367f032d93f642f64180aa3
  Transaction:         0xe9e5f6f06f4999a6deac81327502993eab697d54bb9d8304e27f1a219d45b74e
  From:                0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266
  Value:               0 ETH
  Gas used:            1846796 of 1846796
  Block #1:            0x48a190d5e57376f7e3ca3e5e801d3bff97c761614f45aaa6adf37e851b1ea1a9
[...]

yarn deploy:
[user@TheDesktop end-to-end]$ yarn deploy
yarn run v1.22.10
$ yarn workspace @scaffold-eth/hardhat deploy
$ hardhat run scripts/deploy.js
✖ Help us improve Hardhat with anonymous crash reports & basic usage data? (Y/n) · true

 📡 Deploying...

 🛰  Deploying: NFTMinter
 📄 NFTMinter deployed to: 0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9
 ⛽ 0.014774368 ETH, tx hash 0x704630de8265f85648393fbfd660c9a72bdc761b0f3f80ce89e4299072b0ec13
 💾  Artifacts (address, abi, and args) saved to:  packages/hardhat/artifacts/

$ hardhat run scripts/publish.js
✖ Help us improve Hardhat with anonymous crash reports & basic usage data? (Y/n) · true
 💽 Publishing NFTMinter to ../react-app/src/contracts
 ⚠  Can't publish NFTMinter yet (make sure it getting deployed).
Done in 25.86s.

yarn start which ultimately clears the terminal and outputs:
Compiled with warnings.

./src/components/Swap.jsx
  Line 124:6:  React Hook useEffect has missing dependencies: '_tokenListUri' and 'activeChainId'. Either include them or remove the dependency array  react-hooks/exhaustive-deps
  Line 185:6:  React Hook useEffect has a missing dependency: 'getTrades'. Either include it or remove the dependency array                            react-hooks/exhaustive-deps
  Line 195:6:  React Hook useEffect has a missing dependency: 'exact'. Either include it or remove the dependency array                                react-hooks/exhaustive-deps

./src/hooks/Debounce.js
  Line 14:6:  React Hook useEffect has a missing dependency: 'delay'. Either include it or remove the dependency array  react-hooks/exhaustive-deps

./src/components/NFTViewer.jsx
  Line 1:16:  'Upload' is defined but never used                                                                         no-unused-vars
  Line 1:24:  'Input' is defined but never used                                                                          no-unused-vars
  Line 6:10:  'useParams' is defined but never used                                                                      no-unused-vars
  Line 43:7:  img elements must have an alt prop, either with meaningful text, or an empty string for decorative images  jsx-a11y/alt-text

./src/components/Minter.jsx
  Line 1:39:    'Col' is defined but never used                                                                            no-unused-vars
  Line 75:10:   'tokenId' is assigned a value but never used                                                               no-unused-vars
  Line 110:32:  img elements must have an alt prop, either with meaningful text, or an empty string for decorative images  jsx-a11y/alt-text

./src/hooks/Poller.js
  Line 25:6:  React Hook useEffect has a missing dependency: 'fn'. Either include it or remove the dependency array. If 'fn' changes too often, find the parent component that defines it and wrap that definition in useCallback  react-hooks/exhaustive-deps

./src/hooks/OnBlock.js
  Line 32:6:  React Hook useEffect has missing dependencies: 'args' and 'fn'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

./src/hooks/Balance.js
  Line 35:5:  React Hook useCallback has a missing dependency: 'balance'. Either include it or remove the dependency array  react-hooks/exhaustive-deps

./src/hooks/TokenList.js
  Line 44:6:  React Hook useEffect has missing dependencies: '_tokenListUri' and 'chainId'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

./src/App.jsx
  Line 4:25:   'Col' is defined but never used                                                                                no-unused-vars
  Line 4:36:   'Row' is defined but never used                                                                                no-unused-vars
  Line 11:29:  'Faucet' is defined but never used                                                                                no-unused-vars
  Line 11:37:  'GasGauge' is defined but never used                                                                                no-unused-vars
  Line 11:74:  'Ramp' is defined but never used                                                                                no-unused-vars
  Line 17:3:   'useContractReader' is defined but never used                                                                               no-unused-vars
  Line 18:3:   'useEventListener' is defined but never used                                                                                no-unused-vars
  Line 20:3:   'useExternalContractLoader' is defined but never used                                                                       no-unused-vars
  Line 22:3:   'useOnBlock' is defined but never used                                                                                no-unused-vars
  Line 96:9:   'tx' is assigned a value but never used                                                                                no-unused-vars
  Line 143:6:  React Hook useEffect has a missing dependency: 'localChainId'. Either include it or remove the dependency array             react-hooks/exhaustive-deps
  Line 221:9:  Emojis should be wrapped in <span>, have role="img", and have an accessible description with aria-label or aria-labelledby  jsx-a11y/accessible-emoji

./src/components/Faucet.jsx
  Line 66:5:  React Hook useCallback has an unnecessary dependency: 'props.onChange'. Either exclude it or remove the dependency array  react-hooks/exhaustive-deps

./src/components/ThemeSwitch.jsx
  Line 8:35:  'status' is assigned a value but never used  no-unused-vars

Search for the keywords to learn more about each warning.
To ignore, add // eslint-disable-next-line to the line before.

If you need any additional information, please let me know :).

@yusefnapora Can you please troubleshoot? Thank you!

Gah, I meant to post this to https://github.com/ipfs-shipyard/nft-school-examples, my bad! (Is it fine here?)

Thanks for the report! I’ll dig in and try to reproduce when I’m back at my laptop.

@TheDiscordian I just merged a fix for this to the example repo. Thanks again :)


Cannot drop the table

Hello,
I encoutered an issue using EFCore BulkExtensions.
The db user is set as db_owner on the related database

Does anyone has any clue on that ?

Code I use for that :

If you need any information, do not hesitate.
PS : It's not about rights, I can delete a table with Navicat / SQL Management Studio / MSSQL Tools with the same credentials
Btw, thanks for your awesome work, it's saving my life <3

It happens when I'm updating data from previously recovered entities.
I'll try the sample asap (hope tomorrow)

Another thing, I'm using MSSQL for Linux, on Docker
https://hub.docker.com/r/microsoft/mssql-server-linux/
latest release

If you are doing only Update then use just BulkUpdate, no need for BulkInsertOrUpdate if you don't have new entities.
Secondly UseTempDb is False by default so no need for explicitly setting it to false.
Also if you have only one Bulk operation in method then explicit transaction is redundant.

So far I have only tested it with MSSQL on Win 10.

Hey !
I've got some more informations that I've found !

The exception thrown is not correct (at least, not deep enough), it looks like when a merge fails, the table is automatically dropped (A way to add a custom exception to handle it correctly ?)
The issue does not come from you but it can improve your awesome project !
:)

Great you solved it.
I have just made a test for this situation(InsertOrUpdate-ing with 2 row that have same Id in Source) and it gives me correct error:

Test failed
Message: _System.Data.SqlClient.SqlException
The MERGE statement attempted to UPDATE or DELETE the same row more than once. This happens when a target row matches more than one source row. A MERGE statement cannot UPDATE/DELETE the same row of the target table multiple times. Refine the ON clause to ensure a target row matches at most one source row, or use the GROUP BY clause to group the source rows.

I'm not sure why you first got Can Not Drop the Table error.
I have also checked and MsSQL does not Drop custom Temp tables on its own.
Maybe it could be something specific for MsSQL on Linux but I doubt.

You can make a test with these scripts.
First create EFCoreBulk Test DB:
EFCoreBulkTestDB.sql
Then Run this Sql script.
EFCoreBulkTestSQL.sql
It should give you correct error and then refresh it and check it Temp table remaind in DB.

Sorry for the late reply, I was busy, i'll try it as soon as I arrive home (can't try it with my actual env)


Editor Font not sharp

Iam using windows 7 with vs code November 2019 (version 1.41) ,the editor font is not sharp and is kind of pexelated ,I've added a picture that compares it with Notpad3-mod an editor that uses sharp font text.

@OmarBoshra What font family and font size are you using?

I opened vs code today and it had sharp text ,don't know what happened but if you fixed it ,thank you.


How do I use the mongoose in the resolvers？

I want use mongoose in graphql-tools。
I use Typescript.
my models code:
import * as mongoose from 'mongoose';
interface IBook extends mongoose.Document {
  title: string;
}
const bookSchema = new mongoose.Schema({
  title: String,
});
//mongoose.model(modelName, schema);
export const Book = mongoose.model<IBook>('Book', bookSchema);

my graphql-tools schema code:
import { makeExecutableSchema } from 'graphql-tools';
import { resolvers } from '../resolvers/resolver';

const typeDefs = `
  type Query {
    books: [Book]
  }
  type Book {
    title: String
  }
`;
export const schema = makeExecutableSchema({
  typeDefs,
  resolvers,
});

my graphql-tools resolvers code:
import {Book} from '../models/book';

// The resolvers
export const resolvers = {
  Query: {
    books(title: any) {
      Book.find({ title }, (err, data) => {
        return data;
      });
    }
  }
};

my index.ts code:
import { graphiqlKoa, graphqlKoa } from 'apollo-server-koa';
import { schema } from './schemas/scheme';
import { Book } from './models/book';

router.post('/graphql', graphqlKoa({
  schema
  })
);
outer.get('/graphiql',
  graphiqlKoa({
    endpointURL: '/graphql'
  }
));

then i query books return null.

my mongodb ：

I don’t know where I am having problems。

Your resolver for book makes a call to the Books model and inside the arrow function callback you are returning the data, it is not being captured as the return of the resolver hence the resolver itself implies a return null.
Explicitly performing like so.
  import {Book} from '../models/book';

  // The resolvers
  export const resolvers = {
    Query: {
      books(title: any) {
        Book.find({ title }, (err, data) => {
          return data;
        });
      }
    }
  };

@karloluis Don't you need to return Book.find(...) from the resolver function?

@benjamn That is a possibility, the code I wrote is how the code he wrote is being interpreted by Javascript. As you pointed out, the callback would need it's own return with the expected value. It is not a graph-tools issue.

Thanks everyone,I sloved my problem.I forget async....


Inconsistent $FLOW_CONTEXT with beach exec

Entering the container, running Flow:
$ beach exec
beach@04a76d00f9f2:/application$ ./flow
Neos 8.3.15 ("Development/Beach/Instance" context)

Running Flow directly with beach exec:
$ beach exec ./flow
Neos 8.3.15 ("Development" context)

Running Flow directly with beach exec, passing context:
$ beach exec FLOW_CONTEXT=Development/Beach/Instance ./flow
Neos 8.3.15 ("Development/Beach/Instance" context)

This obviously leads to issues when settings are needed that are specific for Beach…

This should be whatever is configured for the project and definitely the same as when going into the container first.

I think I looked at something similar to this and it's problematic because you can only set the environment for a shell? and with exec direct you are not opening the shell so the environment is never created? Something along those lines.

Well, you can change https://github.com/flownative/localbeach/blob/c11b6efbc8a9a31b7ce850eed8284b9561666ee5/cmd/beach/cmd/exec.go#L49
to be
commandArgs := []string{"exec", "-e", "FLOW_CONTEXT=Development/Beach/Instance", "-ti", sandbox.ProjectName + "_php"}

but with the context potentially configured by the user.

Yep we can do that, good idea! I was mostly saying there is probably no easy way to do that on the container side of things.


panick when use CosineSimilarity

Hi.
With the latest hora, I get a panic when using hora::core::metrics::Metric::CosineSimilarity.
thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/hora-0.1.1/src/core/neighbor.rs:32:54

I think this is an error caused by partial_cmp.
partial_cmp expect it to be None when the given value cannot be ordered.

yes, you are right, because it's float type(https://en.wikipedia.org/wiki/IEEE_754) which have lots of type, such as NaN. it can not be compared, so I have to make sure for every place where has use partial_cmp is valid.
I will fix it as soon as possible and do something to avoid invalid value(maybe just panick?).

hi, I notice a post in Google that has use hora in practice. from the avatar, I think it's your post.
and your application is great👍👍👍, and if you  want to reach hora's full potential, you can enable SIMD support
first use some command, such as lscpu to check which SIMD instruction your platform support, and export an env variable
export RUSTFLAGS='-C target-feature=+avx2,+fma'

and in your Cargo.toml to enable simd feature
hora = {version = "0.1.0", features = ["simd"]}

then, use nightly to run your application, you would find that blazingly fast!
PS: The lack of SIMD-supported documents is my carelessness 😢, I will fill it up immediately, for more information, you can check this documentation

@salamer Thank you.
As you said, I had assumed that SIMD was enabled. It is very helpful.
PS: I published example, and I will do some PR in the future. Thx good project 😄
https://github.com/vaaaaanquish/rust-ann-search-example

Can I display this example link in hora's readme?
I think this is a very complete and great example. The model in my demo is still in Python (huggingface), the others are rust, using RPC to connect. not pure rust 😂😂
Finally, thanks for using hora, I am very happy to see users using it.

Can I display this example link in hora's readme?

Of course. I'm glad :)

I'm continuing to study Japanese, 「小さな恋のうた」 is a song I once learned.

Wow. I was wondering why. It's an interesting connection!

my Japanese level may only be able to read but not the level of daily communication 😂.
and of course, the most direct benefit of learning Japanese is to help the Japanese doc which I am writing 😂.

As far as I can tell, this is because the dot product of a vector and itself is always negative, so the square root is always NaN. Is this intentional? It seems like it is as there is a test in calc.rs:
#[cfg(test)]
mod tests {
    use crate::core::calc::dot;
    #[test]
    fn test_dot() {
        let a = [1., 2., 3.];
        let b = [1., 2., 3.];
        assert_eq!(dot::<f32>(&a, &b).unwrap(), -14.0);
    }
}

And both the simd and non-simd implementations negate the sums:
fn dot_product(a: &[$type_id], b: &[$type_id]) -> Result<$type_id, &'static str> {
                assert_eq!(a.len(), b.len());

                #[cfg(feature = $simd_size)]
                {
                    let size = a.len() - (a.len() % $size);
                    let c = a
                        .chunks_exact($size)
                        .map($simd_type::from_slice_unaligned)
                        .zip(b.chunks_exact($size).map($simd_type::from_slice_unaligned))
                        .map(|(a, b)| a * b)
                        .sum::<$simd_type>()
                        .sum();
                    let d: $type_id = a[size..].iter().zip(&b[size..]).map(|(p, q)| p * q).sum();
                    Ok(-(c + d))  // negated here
                }
                #[cfg(not(feature = $simd_size))]
                {
                    Ok(-(a.iter().zip(b).map(|(p, q)| p * q).sum::<$type_id>())) // negated here
                }
            }

Removing the negation allows cosine similarity to work. Any suggestions about where to go from here?

Great repository! I was able to integrate hora search into a rust application even as an inexperienced rust programmer.
But I cannot use hora::core::metrics::Metric::CosineSimilarity because of the same panic: "called Option::unwrap() on a None value"
What is the status of a fix here?

I'm getting the same crash. Euclidian works, but not Cosine Similarity.


[Shadowrun5thA] calculate drain/nethits for rituals and spirits

Changes / Comments
Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[x] Does the pull request title have the sheet name(s)? Include each sheet name.
[ ] Is this a bug fix?
[x] Does this add functional enhancements (new features or extending existing features) ?
[ ] Does this add or change functional aesthetics (such as layout or color scheme) ?
[ ] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
[ ] If this is a new sheet, did you follow Building Character Sheets standards ?

If you do not know English. Please leave a comment in your native language.

Character Sheet Info Roll20 Internal Use only.

before merging this take a look at https://github.com/Roll20/roll20-character-sheets/pull/9702


Fix Logger constructor

This PR:

Fixes logical check in Logger constructor that was treating LogLevel.Error (with value 0) as falsey and thus always setting level to LogLevel.Info (with value 2) whenever logLevel was configured to Error.
Fixes #3887

Codecov Report

Merging #3899 (aed8869) into dev (21be045) will not change coverage.
The diff coverage is 100.00%.

Flag
Coverage Δ

*Carryforward flag

msal-angular
95.85% <ø> (ø)

Carriedforward from 21be045

msal-browser
85.07% <ø> (ø)

Carriedforward from 21be045

msal-common
85.16% <100.00%> (ø)

Carriedforward from 21be045

msal-core
76.66% <ø> (ø)

Carriedforward from 21be045

msal-node
83.55% <ø> (ø)

msal-react
94.11% <ø> (ø)

Carriedforward from 21be045

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files
Coverage Δ

lib/msal-common/src/logger/Logger.ts
89.39% <100.00%> (ø)


Issues while installing IoT Device SDK v2 for Python on arm64/ubuntu container

I have few questions here.
I am running greengrass inside an arm64/ubuntu:20.0.4 container.
When running greengrass, I tried to provision the device and I was able to see the device on AWS IoT core.
Now to send some sample data to IoT core using greengrass I tried to install python sdk and its dependencies as mentioned in here: https://github.com/aws/aws-iot-device-sdk-python-v2
But when tried installing awscrt dependencies, I got following error:
`root@11b91aae290e:/app# python3 -m pip install awsiotsdk
Collecting awsiotsdk
Downloading awsiotsdk-1.5.17-py3-none-any.whl (44 kB)
|████████████████████████████████| 44 kB 1.1 MB/s
Collecting awscrt==0.11.20
Downloading awscrt-0.11.20.tar.gz (18.3 MB)
|████████████████████████████████| 18.3 MB 9.2 MB/s
Building wheels for collected packages: awscrt
Building wheel for awscrt (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vw3pw14g/awscrt/setup.py'"'"'; file='"'"'/tmp/pip-install-vw3pw14g/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-85qc_jd7
cwd: /tmp/pip-install-vw3pw14g/awscrt/
Complete output (168 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-armv7l-3.8
creating build/lib.linux-armv7l-3.8/awscrt
copying awscrt/mqtt.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/io.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/crypto.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/init.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/exceptions.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/_test.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/auth.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/s3.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/http.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/common.py -> build/lib.linux-armv7l-3.8/awscrt
creating build/lib.linux-armv7l-3.8/awscrt/eventstream
copying awscrt/eventstream/init.py -> build/lib.linux-armv7l-3.8/awscrt/eventstream
copying awscrt/eventstream/rpc.py -> build/lib.linux-armv7l-3.8/awscrt/eventstream
running build_ext
--- Building dependency: aws-lc (RelWithDebInfo) ---
/usr/bin/cmake -DCMAKE_PREFIX_PATH=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install -DCMAKE_INSTALL_PREFIX=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTING=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DBUILD_LIBSSL=OFF -DDISABLE_PERL=ON -DDISABLE_GO=ON /tmp/pip-install-vw3pw14g/awscrt/crt/aws-lc
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
CMake Error at /usr/share/cmake-3.16/Modules/CMakeCompilerIdDetection.cmake:26 (list):
list sub-command REMOVE_ITEM requires two or more arguments.
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:211 (compiler_id_detection)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:230 (CMAKE_DETERMINE_COMPILER_ID_WRITE)
/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake:32 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake:116 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:23 (enable_language)
-- The C compiler identification is unknown
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
pkgconfig not found. Disabling unwind tests.
-- Go not found. Disabling some code generation and using pre-generated code in generated-src/
-- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
Copying platform assembly files from /tmp/pip-install-vw3pw14g/awscrt/crt/aws-lc/generated-src/linux-arm/crypto/ to /tmp/pip-install-vw3pw14g/awscrt/build/deps/aws-lc/crypto
-- Alternative compiler '' detected. Not all flags may be set, check final options with 'cmake --build . -- VERBOSE=1'
-- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-vw3pw14g/awscrt/build/deps/aws-lc/CMakeFiles/CMakeOutput.log".
See also "/tmp/pip-install-vw3pw14g/awscrt/build/deps/aws-lc/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-vw3pw14g/awscrt/setup.py", line 286, in
setuptools.setup(
File "/usr/lib/python3/dist-packages/setuptools/init.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 223, in run
self.run_command('build')
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-install-vw3pw14g/awscrt/setup.py", line 208, in run
self._build_dependency(lib)
File "/tmp/pip-install-vw3pw14g/awscrt/setup.py", line 186, in _build_dependency
subprocess.check_call(cmake_args)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/cmake', '-DCMAKE_PREFIX_PATH=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install', '-DCMAKE_INSTALL_PREFIX=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install', '-DBUILD_SHARED_LIBS=OFF', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DBUILD_TESTING=OFF', '-DCMAKE_POSITION_INDEPENDENT_CODE=ON', '-DBUILD_LIBSSL=OFF', '-DDISABLE_PERL=ON', '-DDISABLE_GO=ON', '/tmp/pip-install-vw3pw14g/awscrt/crt/aws-lc']' returned non-zero exit status 1.
ERROR: Failed building wheel for awscrt
Running setup.py clean for awscrt
Failed to build awscrt
Installing collected packages: awscrt, awsiotsdk
Running setup.py install for awscrt ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vw3pw14g/awscrt/setup.py'"'"'; file='"'"'/tmp/pip-install-vw3pw14g/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-1j4jkvcf/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/awscrt
cwd: /tmp/pip-install-vw3pw14g/awscrt/
Complete output (87 lines):
running install
running build
running build_py
creating build/lib.linux-armv7l-3.8
creating build/lib.linux-armv7l-3.8/awscrt
copying awscrt/mqtt.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/io.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/crypto.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/init.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/exceptions.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/_test.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/auth.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/s3.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/http.py -> build/lib.linux-armv7l-3.8/awscrt
copying awscrt/common.py -> build/lib.linux-armv7l-3.8/awscrt
creating build/lib.linux-armv7l-3.8/awscrt/eventstream
copying awscrt/eventstream/init.py -> build/lib.linux-armv7l-3.8/awscrt/eventstream
copying awscrt/eventstream/rpc.py -> build/lib.linux-armv7l-3.8/awscrt/eventstream
running build_ext
--- Building dependency: aws-lc (RelWithDebInfo) ---
/usr/bin/cmake -DCMAKE_PREFIX_PATH=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install -DCMAKE_INSTALL_PREFIX=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTING=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DBUILD_LIBSSL=OFF -DDISABLE_PERL=ON -DDISABLE_GO=ON /tmp/pip-install-vw3pw14g/awscrt/crt/aws-lc
pkgconfig not found. Disabling unwind tests.
-- Go not found. Disabling some code generation and using pre-generated code in generated-src/
-- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
Copying platform assembly files from /tmp/pip-install-vw3pw14g/awscrt/crt/aws-lc/generated-src/linux-arm/crypto/ to /tmp/pip-install-vw3pw14g/awscrt/build/deps/aws-lc/crypto
-- Alternative compiler '' detected. Not all flags may be set, check final options with 'cmake --build . -- VERBOSE=1'
-- Configuring done
CMake Error at crypto/fipsmodule/CMakeLists.txt:273 (add_library):
Cannot find source file:
    aesv8-armx.S

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
  .hpp .hxx .in .txx

CMake Error at crypto/CMakeLists.txt:218 (add_library):
  Cannot find source file:

    chacha/chacha-armv4.S

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
  .hpp .hxx .in .txx

CMake Error at crypto/CMakeLists.txt:218 (add_library):
  No SOURCES given to target: crypto

CMake Error at crypto/fipsmodule/CMakeLists.txt:273 (add_library):
  No SOURCES given to target: fipsmodule

CMake Generate step failed.  Build files cannot be regenerated correctly.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-vw3pw14g/awscrt/setup.py", line 286, in <module>
    setuptools.setup(
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 144, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/usr/lib/python3.8/distutils/command/install.py", line 589, in run
    self.run_command('build')
  File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-vw3pw14g/awscrt/setup.py", line 208, in run
    self._build_dependency(lib)
  File "/tmp/pip-install-vw3pw14g/awscrt/setup.py", line 186, in _build_dependency
    subprocess.check_call(cmake_args)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/cmake', '-DCMAKE_PREFIX_PATH=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install', '-DCMAKE_INSTALL_PREFIX=/tmp/pip-install-vw3pw14g/awscrt/build/deps/install', '-DBUILD_SHARED_LIBS=OFF', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DBUILD_TESTING=OFF', '-DCMAKE_POSITION_INDEPENDENT_CODE=ON', '-DBUILD_LIBSSL=OFF', '-DDISABLE_PERL=ON', '-DDISABLE_GO=ON', '/tmp/pip-install-vw3pw14g/awscrt/crt/aws-lc']' returned non-zero exit status 1.
----------------------------------------

ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vw3pw14g/awscrt/setup.py'"'"'; file='"'"'/tmp/pip-install-vw3pw14g/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-1j4jkvcf/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/awscrt Check the logs for full command output.`
I also tried to downgrade ubuntu version to 18.0.4 but still the same issue.
Second question is when I installed greegrass software, it is supposed to set few environment variables as described in here: https://docs.aws.amazon.com/greengrass/v2/developerguide/component-environment-variables.html
But it does not set few env variables like:

AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT
SVCUID
I need these environment variables value because I need to create a script which will connect to IPC service and send some data to IoT core.
Can someone tell me what wrong am I doing ?

Please open an issue with the IoT Device SDK for help with building/installing it.

To answer your Greengrass specific question, those environment variables are only available to components which Greengrass launches. If you write a Greengrass component which uses IPC, they will be made available. It is not possible to connect to IPC from an application which Greengrass did not start.

Hey @MikeDombo
Thanks for the reply. I Tried to create a greengrass component but when I tried to run it, it gave me an error saying "could not create IPC client ".
Full logs:
`./bin/greengrass-cli deployment create \

--recipeDir /home/cgt_jpr_pc_nikhil/nikhil/components/greengrassjava/recipes
--artifactDir /home/cgt_jpr_pc_nikhil/nikhil/components/greengrassjava/artifacts
--merge "com.aws.App=2.0.0"
java.lang.RuntimeException: Unable to create ipc client
at com.aws.greengrass.cli.adapter.impl.NucleusAdapterIpcClientImpl.getIpcClient(NucleusAdapterIpcClientImpl.java:260)
at com.aws.greengrass.cli.adapter.impl.NucleusAdapterIpcClientImpl.createLocalDeployment(NucleusAdapterIpcClientImpl.java:172)
at com.aws.greengrass.cli.commands.DeploymentCommand.create(DeploymentCommand.java:90)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at picocli.CommandLine.executeUserObject(CommandLine.java:1952)
at picocli.CommandLine.access$1100(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
at picocli.CommandLine.execute(CommandLine.java:2058)
at com.aws.greengrass.cli.CLI.main(CLI.java:57)
Caused by: java.io.IOException: Not able to find auth information in directory: /home/cgt_jpr_pc_nikhil/greengrass/v2/cli_ipc_info. Please run CLI as authorized user or group.
at com.aws.greengrass.cli.adapter.impl.NucleusAdapterIpcClientImpl.loadCliIpcInfo(NucleusAdapterIpcClientImpl.java:320)
at com.aws.greengrass.cli.adapter.impl.NucleusAdapterIpcClientImpl.getIpcClient(NucleusAdapterIpcClientImpl.java:231)
... 14 more
Suppressed: java.nio.file.NoSuchFileException: /home/cgt_jpr_pc_nikhil/greengrass/v2/cli_ipc_info/user-1001
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
at java.base/java.nio.file.Files.newByteChannel(Files.java:422)
at java.base/java.nio.file.Files.readAllBytes(Files.java:3206)
at com.aws.greengrass.cli.adapter.impl.NucleusAdapterIpcClientImpl.loadCliIpcInfo(NucleusAdapterIpcClientImpl.java:324)
... 15 more
Suppressed: java.nio.file.NoSuchFileException: /home/cgt_jpr_pc_nikhil/greengrass/v2/cli_ipc_info/group-1001
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
at java.base/java.nio.file.Files.newByteChannel(Files.java:422)
at java.base/java.nio.file.Files.readAllBytes(Files.java:3206)
at com.aws.greengrass.cli.adapter.impl.NucleusAdapterIpcClientImpl.loadCliIpcInfo(NucleusAdapterIpcClientImpl.java:330)
... 15 more
Suppressed: java.nio.file.AccessDeniedException: /home/cgt_jpr_pc_nikhil/greengrass/v2/cli_ipc_info/user-0
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
at java.base/java.nio.file.Files.newByteChannel(Files.java:422)
at java.base/java.nio.file.Files.readAllBytes(Files.java:3206)
at com.aws.greengrass.cli.adapter.impl.NucleusAdapterIpcClientImpl.loadCliIpcInfo(NucleusAdapterIpcClientImpl.java:339)
... 15 more
`
Then I checked the environment variables and find these missing.
If you can help me with this

You did not run the CLI as root. You must run it as root, or add yourself to the allowed users in the CLI configuration when you deploy it.

Hey @Mike
I am able to run it as root. But what if I need to run greengrass-cli inside a container. I cannot use sudo inside a container. Then how can I configure to run greengrass-cli to run inside a container..?

You do need sudo inside the container. Greengrass requires sudo to work. Have a look at our Docker guide: https://docs.aws.amazon.com/greengrass/v2/developerguide/run-greengrass-docker.html.
As I mentioned you can also authorize different groups to have access. See https://docs.aws.amazon.com/greengrass/v2/developerguide/greengrass-cli-component.html#greengrass-cli-component-configuration

Hi @MikeDombo
I have one more question here. I was able to run greengrass cli inside the container by authorizing the group to have access to greengrass cli but whenever I try to create a custom component either by greengrass-cli or from AWS IoT core(by adding recipe file and storing jar in S3 buckets), i get following error:
`root@69312e900c86:/var/lib/veea/greengrasspv/app/greengrass/v2# cat logs/com.example.PubSubPublisherJava.log
2021-05-25T08:16:32.550Z [INFO] (pool-2-thread-23) com.example.PubSubPublisherJava: shell-runner-start. {scriptName=services.com.example.PubSubPublisherJava.lifecycle.Run, serviceName=com.example.PubSubPublisherJava, currentState=STARTING, command=["java -jar /var/lib/veea/greengrasspv/app/greengrass/v2/packages/artifacts/com...."]}
2021-05-25T08:16:32.626Z [ERROR] (pool-2-thread-23) com.example.PubSubPublisherJava: shell-runner-error. Error while running process. {scriptName=services.com.example.PubSubPublisherJava.lifecycle.Run, serviceName=com.example.PubSubPublisherJava, currentState=STARTING, command=["java -jar /var/lib/veea/greengrasspv/app/greengrass/v2/packages/artifacts/com...."]}
java.io.IOException: Cannot run program "sudo" (in directory "/var/lib/veea/greengrasspv/app/greengrass/v2/work/com.example.PubSubPublisherJava"): error=2, No such file or directory
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at java.base/java.lang.Runtime.exec(Runtime.java:592)
at com.aws.greengrass.util.Exec.exec(Exec.java:422)
at com.aws.greengrass.util.Exec.background(Exec.java:469)
at com.aws.greengrass.lifecyclemanager.ShellRunner$Default.successful(ShellRunner.java:102)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.run(GenericExternalService.java:571)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.run(GenericExternalService.java:514)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.handleRunScript(GenericExternalService.java:352)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.startup(GenericExternalService.java:343)
at com.aws.greengrass.lifecyclemanager.Lifecycle.lambda$handleStateTransitionStartingToRunningAsync$9(Lifecycle.java:531)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: error=2, No such file or directory
at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
at java.base/java.lang.ProcessImpl.(ProcessImpl.java:340)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
... 15 more
2021-05-25T08:16:32.648Z [INFO] (pool-2-thread-24) com.example.PubSubPublisherJava: shell-runner-start. {scriptName=services.com.example.PubSubPublisherJava.lifecycle.Run, serviceName=com.example.PubSubPublisherJava, currentState=STARTING, command=["java -jar /var/lib/veea/greengrasspv/app/greengrass/v2/packages/artifacts/com...."]}
2021-05-25T08:16:32.719Z [ERROR] (pool-2-thread-24) com.example.PubSubPublisherJava: shell-runner-error. Error while running process. {scriptName=services.com.example.PubSubPublisherJava.lifecycle.Run, serviceName=com.example.PubSubPublisherJava, currentState=STARTING, command=["java -jar /var/lib/veea/greengrasspv/app/greengrass/v2/packages/artifacts/com...."]}
java.io.IOException: Cannot run program "sudo" (in directory "/var/lib/veea/greengrasspv/app/greengrass/v2/work/com.example.PubSubPublisherJava"): error=2, No such file or directory
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at java.base/java.lang.Runtime.exec(Runtime.java:592)
at com.aws.greengrass.util.Exec.exec(Exec.java:422)
at com.aws.greengrass.util.Exec.background(Exec.java:469)
at com.aws.greengrass.lifecyclemanager.ShellRunner$Default.successful(ShellRunner.java:102)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.run(GenericExternalService.java:571)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.run(GenericExternalService.java:514)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.handleRunScript(GenericExternalService.java:352)
at com.aws.greengrass.lifecyclemanager.GenericExternalService.startup(GenericExternalService.java:343)
at com.aws.greengrass.lifecyclemanager.Lifecycle.lambda$handleStateTransitionStartingToRunningAsync$9(Lifecycle.java:531)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: error=2, No such file or directory
at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
at java.base/java.lang.ProcessImpl.(ProcessImpl.java:340)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
... 15 more
`
Not sure what am I missing here. I am able to create and deploy custom component on my local ubuntu machine but when trying same thing inside my container, I am getting the above errors.
When I try to check component list, I get my component in broken state:
Components currently running in Greengrass: Component Name: FleetStatusService     Version: null     State: RUNNING     Configuration: null Component Name: aws.greengrass.Nucleus     Version: 2.1.0     State: FINISHED     Configuration: {"awsRegion":"us-east-1","componentStoreMaxSizeBytes":"10000000000","deploymentPollingFrequency":"15","deploymentPollingFrequencySeconds":15.0,"envStage":"prod","fleetStatus":{"periodicStatusPublishIntervalSeconds":86400.0},"greengrassDataPlanePort":"8443","iotCredEndpoint":"c2xd20u9384ny8.credentials.iot.us-east-1.amazonaws.com","iotDataEndpoint":"a2edtrcjia9l8j-ats.iot.us-east-1.amazonaws.com","iotRoleAlias":"GreengrassV2TokenExchangeRoleAlias","jvmOptions":"-Dlog.level= -Dlog.store=FILE","logging":{},"mqtt":{"spooler":{}},"networkProxy":{"proxy":{}},"platformOverride":{},"runWithDefault":{"posixShell":"sh","posixUser":"ggc_user:ggc_group"},"telemetry":{}} Component Name: TelemetryAgent     Version: 0.0.0     State: RUNNING     Configuration: null Component Name: aws.greengrass.Cli     Version: 2.1.0     State: RUNNING     Configuration: {"AuthorizedPosixGroups":"ggc_group"} Component Name: DeploymentService     Version: 0.0.0     State: RUNNING     Configuration: null Component Name: UpdateSystemPolicyService     Version: 0.0.0     State: RUNNING     Configuration: null Component Name: com.example.PubSubPublisherJava     Version: 1.0.0     State: BROKEN     Configuration: {"accessControl":{"aws.greengrass.ipc.mqttproxy":{"com.example.PubSubPublisherJava:pubsub:1":{"operations":["aws.greengrass#PublishToIoTCore"],"policyDescription":"Allows access to publish to all topics.","resources":["*"]}}}}

The problem is that your container is not configured with the sudo program. Please follow our docker guide to setup a docker image properly: https://docs.aws.amazon.com/greengrass/v2/developerguide/run-greengrass-docker.html


No connection established

We tried CritParty today with two people and couldn't establish a connection.
After an initial error message "A tab needs to be open" nothing seemed to work.
We tried both ways, trying to host and trying to join a session but nothing happened.
There were no further pop up error messages, nor messages in the CritParty Window or the output window.
We both use Glyphs 2.6.4 (1286) which according to Glyphs is up to date.

Can you update to the latest cutting version (activate it in Preferences > Updates).

We both did update to Version 2.6.5 (1339) but somehow there is still nothing happening.
Also no error messages I could share.

Daft question but... did you open a tab with some glyphs in it before trying again?

Also please try uninstalling, downloading the debug version from http://www.corvelsoftware.co.uk/software/CritParty.glyphsPlugin-debug.zip and trying again; then attach the log files from Documents/webrtc_logs/ on each computer.

Daft question but... did you open a tab with some glyphs in it before trying again?

Yes I did. At first I was just in the overview but then I got a pop up error message and then of course I tried it with an open tab with a glyph in it, but that didn’t help.

Also please try uninstalling, downloading the debug version from http://www.corvelsoftware.co.uk/software/CritParty.glyphsPlugin-debug.zip and trying again; then attach the log files from Documents/webrtc_logs/ on each computer.

We will as soon as we find the time again, thanks for the debug version :)

A friend and I were unable to establish a connection.
At first, we were on the different version, 2.6.6 (1343) and 2.6.5 (1342), then we tried on the same 2.6.5 (1342) version. In each case, it wasn't working.
One thing that was strange was that whenever my friend would share a session ID, I would click connect and nothing would happen and no popup error messages. Whenever I would create a session ID and my friend tried to join, Glyphs would immediately crash and they would get the message "Host Closed Connection Party's over, folks".
We were also unsure what should be the setup on each of our computers. Should only one person have a glyphs file open the other only glyphs open with no file loaded? Will a new window open for the other person? Should the other person have a blank file open? Should both had the same file?
We tried multiple scenarios but no success.
Any suggestions on what we can try?
If it's a firewall issue, how can we check that?

That's very weird. Could you try the debug version linked above? The host user needs to have a glyphs file open (the plugin will shout at them if they don't) and it doesn't matter about the guest user.

We tried again with a new install of the debug version on Glyphs Version 2.6.5 (1342).
First time I tried as Host with one tab open. Friend had nothing open. Result was that Glyphs crashed.
Second time my friend tried as Host and I had nothing open. Result was that nothing happened when I click join.
I've emailed you both of webrtc_logs and the glyphs error I received.

Brilliant. Thanks. I shall take a look.

Aargh, it's this problem: https://groups.google.com/forum/#!msg/discuss-webrtc/P4vVIvPL7Tg/nDxXaxjbCQAJ
Strings are getting lost within the WebRTC library, somewhere between Objective C and C++. And I have no idea how to solve it. :-(

Arg! Hopefully, there is some feedback on the forum on how to solve it.
If there is an update, feel free to contact me and to help test it.

The c++ <> objectiveC Problem sounds like a memory issue. How do you convert the string. Maybe the source string is deallocated before the target gets an hold of it?

The strings are being created here: https://github.com/simoncozens/CritParty/blob/d8dcd54549acceab4d680493e121bf86c4d60b04/CritParty/CritParty/CritParty.m#L433-L438
and then gets to C++ through this code here: https://chromium.googlesource.com/external/webrtc/+/branch-heads/62/webrtc/sdk/objc/Framework/Classes/PeerConnection/RTCPeerConnection.mm#216

The strings are all in the source, so they shouldn’t be deallocated. I don’t understand the code, yet. Can you run it in Xcode with enabled memory protection and zombie settings?

How do I debug a plug-in in Xcode?

Has CritParty been updated for Glyphs 3?
Tried using the plugin again with a friend but we were still running into connection issues.
As soon as either one of us tried to Join by entering the Session ID and Password, it immediately disconnects and says something along the lines of "the party is over".
On Glyphs 3 Cutting Edge 3.0.5 (3131) with installing the latest version of the plugin


Update poetry.lock

There seems to be a missing dependency xcode for psutil on the macos Nix build: https://github.com/runtimeverification/pyk/actions/runs/6665614122/job/18115790546?pr=689#step:5:2669
@goodlyrottenapple, is it something we need to address ourselves, or should we just wait until some upstream flake is fixed? (This is low priority.)

Looks like there has been a recent change: https://github.com/nix-community/poetry2nix/commit/26e75e526d1ce5859b7cd1b70ef96c6ba1085e4e
Unfortunately i cannot reproduce on my m1 mac, looks to be intel mac specific

I do indeed; will take a look at reproducing locally on this branch


import of '!text' file via plugin-text module generates error

We are using the plugin-text module to allow importing of text as follows:
import template from './template.html!text';

However, this is producing the following error when running flow against the code:
  4: import template from './template.html!text';
                          ^^^^^^^^^^^^^^^^^^^^^^ ./template.html!text. Required module not found

Any suggestions on how to handle this?
One solution would be to switch the flow checking off for a code block (using comments, like you can in jshint/jslint etc) – I searched the flowtype docs, but couldn't find that capability there (did I miss something?!).

I think you can create a new file, say flow_text.js with contents like
// @flow
module.exports = 'some string';

and redirect modules ending with !text to this file using name_mapper.

Thanks @agentcooper.
In my case, I ended up using the templateUrl property instead, as it is being used for an Angular component's template.

For anyone else running into this, see specific solutions in MoOx’s gist.
Also, pretty sure this issue can be closed.


Implement u8 load size with double read

To have u8 load size work transparently with decoding tables of at most 16 bits we need to load with size u8 twice. This will need some conditional compilations (all other loads happen once).

Probably not worth looking into.