Invalid HTTP request received, websocket

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the FastAPI documentation, with the integrated search.
[X] I already searched in Google "How to X in FastAPI" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to FastAPI but to Pydantic.
[X] I already checked if it is not related to FastAPI but to Swagger UI.
[X] I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code
import string
from typing import Dict
from fastapi import WebSocket

class ConnectionManager:
    def __init__(self):
        self.active_connections: Dict[str,WebSocket] = {}

    async def connect(self, websocket: WebSocket, userid: str):
        await websocket.accept()
        self.active_connections[userid]=websocket

    def disconnect(self, userid: str):
        del self.active_connections[userid]

    async def broadcast(self, data):
        for k in self.active_connections.keys():
            await (self.active_connections[k]).send_json(data)

    async def get_active_users(self):
        return [a for a in self.active_connections.keys()]

Description
Whenver i try to connect to websocket i get  Invalid HTTP request received.
Operating System
Windows
Operating System Details
win11
FastAPI Version
0.70.1
Python Version
Python 3.10.1
Additional Context
it was working about a year ago,i was just updating the project.

Had the same issue on linux with firefox.
I doubt its a problem with fastapi or startlette but appears to be an issue with websockets itself or uvicorn.
Try specifying the "WebSocket protocol implementation" argument -ws on uvicorn as wsproto:
uvicorn main:app --ws  wsproto
Hope that helps!


changes for beta5

Prep for beta5.

Remove DeepViolet UI and command line reference cases.  Will move those to another project later.  DeepViolet project will hold the code for the API only.
Add changes to support automatic generation of API jar and publish to Maven Central.  I'm not sure how well this will work since the only way is to test in place.  I will need a few cycles to get it right i'm sure.

@spoofzu are there missing changes in pom.xml? The current groupId (as in master) might be a problem to deploy to Maven Central as the domain seems to be owned by a 3rd-party(?). [1]
[1] http://central.sonatype.org/pages/choosing-your-coordinates.html

Missing souce code changes

Thanks @thc202,  I’m git challenged.  I noticed some of the errors indicated that all my changes were not pushed.  I need to try this again.  After I get my code in I’m sure I will need some tuning on the Travis/Maven side.  I want it to push to maven central automagically every time I merge a pull to master.  I’m a good enough Java developer but getting this systems side of the house working is frustrating for me but It’s good information to know.  I’m sure I will make it work but not without falling on my face a few more times.  Thanks for tracking the project.
—Milton

On Nov 23, 2016, at 8:44 AM, thc202<EMAIL_ADDRESS>wrote:
@spoofzu https://github.com/spoofzu are there missing changes in pom.xml? The current groupId (as in master) might be a problem to deploy to Maven Central as the domain seems to be owned by a 3rd-party(?). [1]
[1] http://central.sonatype.org/pages/choosing-your-coordinates.html http://central.sonatype.org/pages/choosing-your-coordinates.html
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/spoofzu/DeepViolet/pull/25#issuecomment-262568248, or mute the thread https://github.com/notifications/unsubscribe-auth/AIDyN6Pheytdb2C07y407xePH7oZhwPmks5rBG1ygaJpZM4K6wYv.


[DOCFIX] - TODOs for UFS API extensions

@calvinjia @aaudiber

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/497/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/499/
Test FAILed.


[Question]: [Bug]: Fetching user's coordinates working fine in debug mode but we launch app in release mode or built release apk the coordinates of the user's not fetched

Please check the following before submitting a new issue.

[X] I have searched the existing issues.
[X] I have carefully read the documentation.

Please select for which platform(s) you need help

[X] Android
[ ] iOS
[ ] Linux
[ ] macOS
[ ] Web
[ ] Windows

Your question
Future<void> onStart(ServiceInstance service) async {
  // Only available for flutter 3.0.0 and later
  DartPluginRegistrant.ensureInitialized();

  final FlutterLocalNotificationsPlugin flutterLocalNotificationsPlugin =
      FlutterLocalNotificationsPlugin();

  Timer.periodic(const Duration(seconds: 10), (timer) async {
    AuthController authController = AuthController();
    bool? isAuthFetched = await authController.loadAuthData();
    if (isAuthFetched == true) {
      ManageEvents manageEvents = ManageEvents();
      if (authController.token != null && authController.token?.value != null) {

        Position position = await Geolocator.getCurrentPosition(desiredAccuracy: LocationAccuracy.low);

        List<UserAcceptedRequests> listUserAcceptedRequests = await manageEvents
            .listUserAcceptedRequests(authController.token!.value);
        listUserAcceptedRequests.forEach((element) {
          //print(element.dateTime);
          int difference = DateTimeUtil.calculateTimeDifferenceInMinutes(
              DateTime.now(), element.dateTime);
          //print(difference);
          if (difference.abs() <= 45987) {
            // TODO: YOU NEED TO CHANGE THE MAX DURATION RANGE
            manageEvents.updateLocationToEventConnectedCrew(
                authController.token!.value,
                element.id,
                position.latitude,
                position.longitude);
            print("Live location updated");
          } else {
            print("No need to update location for now");
          }
        });
      } else {
        print("User not logged in");
      }
    } else {

   // Check initial connectivity
   // Get.put(NetworkService()).checkInitialConnectivity();
    print("Failed to fetch auth");
    }
  });
}

### Version

geolocator: ^10.1.0

Dear @Vaibhav77e,
Can you check if you entered the correct permissions in the manifest and if you requested the correct permissions in the release app?
Kind regards,

Yes i solved it I have called  function  in wrong place that's why it was not working in release mode


feature: add crumbIssuer option to jenkins and improved the UI

…improve the UI to show a notification  after executing the action: re-build
Hey, I just made a Pull Request!

:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[x] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Markdown check failed: Did you really mean 'jenkins'?' How can I fix it? :)

@nodify-at great work! Thinking the color of the notification should probably be green if the task was executed successfully. Red tells me something went wrong 😀

@regicsolutions sorry, I've uploaded this SS by mistake :) I uploaded the correct one, you can check in the code, I'm setting the severity to 'success' for this message :)

@regicsolutions sorry, I've uploaded this SS by mistake :) I uploaded the correct one, you can check in the code, I'm setting the severity to 'success' for this message :)

Looks good!

Must I fix this check issue (it shows a warning about Jenkins word?

I think you might want want to change it to Jenkins or if that doesn't work you could add the word to .github/vocab.txt

I think you might want want to change it to Jenkins or if that doesn't work you could add the word to .github/vocab.txt

fixed :)


pylint version parsing doesn't like the deprecation warning in pylint 1.6.5

pylint 1.6.5 seems to have started outputting a deprecation warning to stderr:
$ pylint --version
pylint 1.6.5,
astroid 1.4.9
Python 3.6.0 (default, Dec 24 2016, 08:02:28)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]
This option 'required-attributes' will be removed in Pylint 2.0This option 'ignore-iface-methods' will be removed in Pylint 2.0

...that last line goes to stderr, but it seems to be read in by the pylint syntastic plugin and causes it to fail.
:messages
syntastic: error: checker output:
This option 'required-attributes' will be removed in Pylint 2.0This option 'ignore-iface-methods' will be removed in Pylint 2.0pylint 1.6.5,
astroid 1.4.9
Python 3.6.0 (default, Dec 24 2016, 08:02:28)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]
syntastic: error: checker python/pylint: can't parse version string (abnormal termination?)

This is a problem in your pylintrc, not in syntastic.  With a clean install:
$ virtualenv -v -p python3 virt
[...]
$ cd virt/
$ source bin/activate
(virt)
$ pip install pylint
[...]
$ pylint --version
pylint 1.6.5,
astroid 1.4.9
Python 3.6.0 (default, Jan 11 2017, 08:45:14)
[GCC 5.4.0]
(virt)

Remove --required-attributes from your pylintrc, and you'll be fine.


Fixed various bugs with range request implementation

Closes #318

Additionally adds a header verfication checker for Content-Length to
ApplicationController
Fixes multiple bugs in range request

@cofiem pretty please review - this patch did my head in. There's beer in it for you 😉 🍺

Coverage decreased (-0.01%) to 85.128% when pulling 02a215820734de7c012b966f2a0d1598d97dbd84 on bug-range-requests into c1e483a18ce2e9b37de81bab4a0b09b0eade2ff3 on develop.

Coverage decreased (-0.04%) to 85.1% when pulling 1145f8f2b2781d6f2459ee6188158a41cc5c4eed on bug-range-requests into c1e483a18ce2e9b37de81bab4a0b09b0eade2ff3 on develop.


Prevent Race Conditions by Utilizing Transaction Isolation

We need to ensure that we don't get non-repeatable reads and phantom reads on some of the API endpoints. In particular anything that Bandersnatch is going to use that has a serial number we need to make sure that nothing changes between the query that gets the serial number and the quer(y|ies) that get the actual data.

Ok, so I've been experimenting with this and I've come across a few problems.

SQLAlchemy ORM doesn't make it particularly great to use a SERIALIZABLE READ ONLY DEFERRABLE transaction.
When using the Pyramid context objects, there's no "hook" that allows us to create a session using information that is specific to the view or route.

These can be worked around, in the first case it can be worked around by doing:
engine = engine.execution_options(isolation_level="SERIALIZABLE")

@event.listens_for(engine, "begin")
def _set_read_only(conn):
    conn.execute("SET TRANSACTION READ ONLY DEFERRABLE")

This has the downside that it requires an extra query in order to actually enable the read only option. I've verified that this works to disable writing to this transaction however I'm unsure if the DEFERRABLE attribute will take effect in this scenario. If it doesn't it's not the end of the world, however it does mean that, if it doesn't, PostgreSQL cannot optimize away maintaining open locks while the transaction is open.
In the second case, this can be worked around by having a read only decorator that does something like:
def read_only(view):
    @functools.wraps(view)
    def wrapped(context, request):
        # Stash the original session
        original_session = request.db

        # We need to create a new session for this request, this session must
        # be read-only.
        session = _create_session(request, read_only=True)

        # Now we want to merge the context object (if we have one) with the
        # new session.
        if (context is not None and
                isinstance(context, ModelBase) and
                _Session.object_session(context) is original_session):
            context = session.merge(context)

        # Now that we've gotten a new session we want to assign it to
        # request.db so that the view will use it instead of the original
        # session.
        request.db = session

        return view(context, request)

    return wrapped

This works as far as I can tell, however I'm a bit concerned about this solution to the problem:

This relies on some bits of SQLAlchemy that I don't fully understand.
I don't know if merging in the session here will actually have the desired effect, since I think it's going to pull things in by primary key it's not going to do any lookups to make sure that we're getting the right value, I think it'll just do a PK lookup.
This relies on an extra transaction in order to enable the read-only bits. Database queries are not the most expensive things in the world, but they aren't free either.
It requires a view to opt-in to this handling, which makes it easy to not get it by accident, however the downsides here are basically we might get an inconsistent response which isn't the worst thing in the world. It causes some bugs in bandersnatch though.

Thinking about this more, I'm wondering if maybe we don't just want to use SERIALIZABLE queries for PyPI always. This isolation level will make PyPI function as if the queries were executes serially and not concurrently. The major downside being that it makes it more likely that if you have two concurrent queries one of them will fail and we'll need code to handle that fact and to do something to retry the transaction. The major upside is that we eliminate any class of bug that stems from non-repeatable reads and phantom reads. These bugs are not normally a big deal however they were a cause of problems on PyPI legacy already that caused me to implement the ability to set a particular transaction to SERIALIZABLE READ ONLY DEFERRABLE.
The biggest downside to SERIALIZABLE is the fact that your application needs to be prepared to retry transactions. Luckily since we're using pyramid_tm we get this "sort of" free because we can configure that with a set number of attempts (defaults to just 1) where if the transaction fails it'll simply re-execute the view and try to commit the transaction again. This can cause problem if the view has side effects that aren't tied into the transaction system, but ideally we tie everything we can into the transaction system.
In addition I believe the risks themselves should be fairly small because for PyPI the read:write ratio is heavily skewed towards reads. In particular, on an example day in February there were only 560 total new releases on PyPI in the entire day. Obviously that's not the entirety of the write operations on PyPI however in general I think the chances of conflict are going to be fairly small because of how unlikely it is we have two things trying to make modifications to the same row.
I'm not the greatest expert at databases here though, there might be some downside to SERIALIZABLE that I'm not aware of.

To be clear, the fundamental problem we have here is that we have views that (essentially) look like this:
# This part is actually outside of the view, and we can't easily change it in a view specific way.
project = sqla_session.query(Project).filter(Project.name == name).one()
# This part *may* by outside of the view, depending on which view it is.
release = sqla_session.query(Release).filter((Release.project == project) & (Release.version == version)).one()
# This part is absolutely inside of our view
serial =  (
    sqla_session.query(func.max(JournalEntry.id))
                .filter(JournalEntry.name == file_.name)
                .scalar()
)

In between any of these three queries the state of the database can change and we can get inconsistent results. This is most noticeable with the serial value, where if something happens between fetching that and the other queries we can get a serial number that doesn't match everything else however we have systems which depend on that matching the current state. Changing the order doesn't help because all that does is mean instead of getting a newer serial than we expected we get an older serial.

Ok, I now have two different solutions:
Solution 1: Read Only Route Predicate
This is the solution that is implemented in #437.
Upsides:

No code or conceptual changes to any views except for ones that need to be read-only, even then those views are just read only with a consistent view of the database.

Downsides:

Can't look at a view and determine if it's read-only or not, you need to look at the route that was given to it.
When not using the Pyramid machinery to execute a view, view won't be read-only (such as in unit tests).
Abuse of the route predicate system to act as a "hook" in the right place.
Multiple SQLAlchemy engines (sharing the same pool though).

Solution 2: SERIALIZABLE all the things
This is the solution implemented in #436.
Upsides:

Consistent behavior across all views.

Downsides:

Views might need to be executed more than once (handled automatically) so all views needs to tie state modification to the transaction system (or otherwise gracefully handle re-execution).

Question about the "fundamental problem" you describe in the three parts, how come these won't all be in the same transaction?

They will be in the same transaction, but the default isolation in PostgreSQL is READ COMMITTED. That means that each individual query gets it's own consistent snapshot of the database that includes everything that's been committed + everything that's changed in this particular transaction. So that means if another transaction changes the state of the database between the execution of these queries and commits it, then our transaction will see that since it's READ COMMITTED.
One of the solutions here is to switch the default isolation level to SERIALIZABLE which is the strictest isolation mode, and it essentially means that the database will take a snapshot at the beginning of the transaction and it won't update what data you can see inside that transaction when other queries commit. This means that if two concurrent transactions touch the same rows, one of them is going to raise an error.

Talked this over with @r1chardj0n3s, we've decided to go with #436 since that'll ensure every transaction has a consistent snapshot of the database and we'll consider any view that can't handle pyramid_tm doing a retry as a bug.

Would be fascinated to know what conditions READ COMMITTED won't be enough...

We have a data model where we essentially have one table that we uses as an append only log of all of the meaingful actions that have been done to the database and we use the ID column of that table as a sort of "this is the version of the data we are currently in". We have mirrors that replicate the PyPI data and they use that ID number so they know, given the list of changes from the audit log, what changes they've applied and what they haven't.
So the basic problem is this
Transaction 1             | Transaction 2             |
SELECT ... FROM packages  |                           | audit_log version is 1
                          | INSERT INTO audit_log ... | audit_log version is 2
SELECT ... FROM audit_log |                           | audit_log version is 2

So you can see, we do two queries in transaction 1, the first one we select some data from the database and in the second one we get what "version" the data is in by select from the audit_log. However in between these two something has INSERT'd an extra row into the audit_log so while the audit_log version was 1 when we selected the data, we'll see it as 2 when we do the selection.
This causes problems because if we select from the audit_log after selecting the data, then the mirroring system thinks that it's processed audit log entries which it actually hasn't. However if we select from the audit log first then the mirroring system will think it hasn't processed some audit log entries even though it actually has.

Did you have a look at  SELECT FOR UPDATE? Either way, for another set of very valuable eyes on this, I'd suggest asking on the sqlalchemy mailing list, @zzzeek will almost certainly have wise words of wisdom :-)


added High Integrity Coding Standard

Programming Research also provides static code checkers (i.e. QA-C++).  Their coding standard is one of the contributions for MISRA C++ used extensively in the automotive industry.

Thank you!


Uncompressed CSV

Allow to output uncompressed CSV which will help for compatibility with bragi importer.

BTW I was thinking: I wrote a small function to have the time displayed before a message. Maybe I should move it to the common library so we use it everywhere? What do you think?

BTW I was thinking: I wrote a small function to have the time displayed before a message. Maybe I should move it to the common library so we use it everywhere? What do you think?

It could indeed be a nice thing


Running Multiple Build is creating nested hierarchy in deployment folder

Describe The Bug
Whenever I am trying to run npm run deploy for 2nd time onwards on a project, it's creating a nested hierarchy of deployment folder inside the deployment folder. This is breaking the path and cf deployment is not able to find main.ts
I have 3 projects. 2 of them are working fine but this behavior is occurring for only one project. Please find the folder structure generated in the attached image.
Every time when I have to deploy the application, I have to delete the previous build.
1st build-

2nd build- You can see there's a deployment under the deployment folder.

To Reproduce
Steps to reproduce the behavior:

run npm run deploy 2 times.

Expected Behavior
It should deploy the application without error, we should not delete the previous build manually.
Used Versions:

@sap-cloud-sdk/cli/0.1.10 win32-x64 node-v12.16.1
npm: '6.13.4'

Folder structure of the application where the build is failing

Hello @gopalanand333,
I can't reproduce the bug so I have a few questions:

What is your deployment script? (it's located in package.json under "scripts")
Is your deployment folder inside the project folder?

Hi @CharlesDuboisSAP,

Deployment script:   "deploy": "npm run build && sap-cloud-sdk package && cf push". I am using npm run deploy
Yup, my deployment folder is under the project folder. While building I can see the dist folder is deleted and generated again but the deployment folder is having different behavior.

One point to note is that it's working fine for 2/3, just one is failing.

Good morning,
could you perhaps share the failing project so that we have an example to reproduce the error.
Best
Frank

@gopalanand333
You added "allowJs": true in your tsconfig.json file which looks for all js files, even the ones in the deployment folder and puts them again in the dist folder.
Here are the solutions:

Delete this option from the tsconfig.json file. You only have ts files in your source folder, you don't need to look for js files.
Add the deployment folder to the exclude list in tsconfig.build.json
Add an include list the same way as the exclude list that only looks in the src folder

In the long run we will probably exclude the deployment folder or only include the src folder in order to allow the user to add "allowJs": true without it making nested folders.

@CharlesDuboisSAP
Thanks for providing these solutions.
Deleting the "allowJs": true worked.
People might use js files in the future, it will be good to have the deployment folder in exclude list by default in the future.
Best regards,
Gopal


Bad request (status code 400) is not really representative of the error

Hi,
I tried using Direct API and targeted "Create a Feature" API.
I point that adding a "feature_1" works fine, adding a second feature "feature_2" works fine, but adding the same existing feature "feature_1" returns

django.request WARNING  Bad Request: /api/v1/projects/1/features/

I think this is misleading, probably same "Bad Request error, 400" but better with an message about "duplicates".
https://stackoverflow.com/a/3290198
Thanks :)
gz#380

Hi @bacloud23, you can see the detail of the error message in the response from the API (as per screenshot below). This warning message in the logs is regarding the request itself. As it's a handled error message, and a response is provided to the client, I don't see the need to provide more information in the server logs. I'm going to go ahead and close this issue based on this.

100% my bad, I didn't pay attention to the response object .


Fix Defaults for PV and/or Battery

Method:

Open a Model with a config
Shift+F10
Change to another Model
Restore inputs to defaults
Shift+F10
Repeat above 3 all configs
Use Gitkraken or some visual Git editor to check changes

Most fixes are around cost defaults, but some are related to default size of the battery system
Close ##665

@dguittet @cpaulgilman So I don't think the listed methods had the desired effect. I think using Change Model followed by Reset inputs to default values is keeping the defaults of the original configs, so the system capacities and costs from whatever the first config was were written to the remaining configs. Hence the failing Travis tests. I'm gonna try to go 1 config at a time to see if that has the intended effect.

Closing as this was addressed in #668


feat: text color options in bubblemenu

Text color drop

@rowheat02 love this – thanks for the PR!
Quick note: some of the colors are a bit too bright/light for a white background – can we make sure that all the colors pass WCAG 2.0 contrast requirements please?

@steven-tey updated with passed color codes.

Merging this now – thank you so much @rowheat02! 🙏


CA e-bike incentives project revised labels json file

Hi @shankari, I'm submitting a new pull request for the revised json file for mode and trip purpose labels for the Open Path app for the CA e-bike incentives project - in the last project meeting, the team revised the mode and purpose labels. This revised json file reflects these changes. The revised trip purpose labels are hopefully short enough to fit a phone screen. Please let me know if you have any questions. I'm CCing @dtfitch in this github thread. Thank you!

Thanks Shankari. I don't think we have a Spanish speaker in our team.

@Abby-Wheelis can you add the Spanish translations to this when you have a chance?


Add OccupancyGrid to ros-bridge

Description

Where has this been tested?

Platform(s): Ubuntu 16.04
Python version(s): Python 2.7
Unreal Engine version(s): ...

This change is

Should I need to do some works on PR? There's some FIXME comments, but they are related to other parts of CARLA, so I'm not sure how to fix them.

Review status: 0 of 3 files reviewed, 2 unresolved discussions (waiting on @danil-tolkachev)

carla_ros_bridge/src/carla_ros_bridge/map.py, line 55 at r1 (raw file):

       # FIXME: height for Town01 is still in centimeters

Maybe create an issue on carla board ?

carla_ros_bridge/src/carla_ros_bridge/map.py, line 60 at r1 (raw file):

    def send_map(self):
        self.map_pub.publish(self.map_msg)

Here the map will not be saved in the rosbag.  Do we need to add a way to save a kind of latch topic in the rosbag (i.e only once), or should we keep the map only for real time processing/not replay.
The user can still use rosbag record -a , to create a rosbag, but it introduce unsynch behavior..

Comments from Reviewable

Review status: 0 of 3 files reviewed, 2 unresolved discussions (waiting on @danil-tolkachev)

carla_ros_bridge/src/carla_ros_bridge/map.py, line 55 at r1 (raw file):
Previously, laurent-george (George Laurent) wrote…

       # FIXME: height for Town01 is still in centimeters

Maybe create an issue on carla board ?

I opened #541

carla_ros_bridge/src/carla_ros_bridge/map.py, line 60 at r1 (raw file):
Previously, laurent-george (George Laurent) wrote…
Here the map will not be saved in the rosbag.  Do we need to add a way to save a kind of latch topic in the rosbag (i.e only once), or should we keep the map only for real time processing/not replay.
The user can still use rosbag record -a , to create a rosbag, but it introduce unsynch behavior..

I didn't work with CarlaRosBridgeWithBag yet. @laurent-george, by unsynch behavior do you mean that it is possible that not all messages will be сought in a bag file? In that case I think that it would be nice to support map in the bridge with bag too.

Comments from Reviewable

@laurent-george could you get some advices how to add map into bridge with bag?

@danil-tolkachev yeah sure, I am looking into it. We can also merge your code without this feature and add a new pull request later. Give me some hours I try to figure out an approach.

@danil-tolkachev I tried to test it on my computer, but I get issue with the map name scene.map_name return an empty string here, thus the .txt is not found.
Have you other change like new settings.yaml not in the pull request ?

Hi @laurent-george. Probably I haven't. I'm not sure, but it seems that you use not the latest version (0.8.4) of Carla. Can you check this?

@nsubiron I didn't understand why the travis build failed here, could you help me on that ?

For some reason, PythonClient/carla/planner/city_track have trailing newlines. I don't know how @felipecode managed to overpass the travis, but it's there 😂
Maybe he can fix it!

@danil-tolkachev can you rebase your branch so it can be merged ?

@laurent-george I've added some changes. Could you review them?

@nsubiron, @marcgpuig
Should I need to do some works on PR? There's some FIXME comments, but they are related to other parts of CARLA, so I'm not sure how to fix them.

Hi, I am a beginner from carla. I just want to integrate a self defined Lidar in Carla system. But the release and rosbridge conbined not correctly. So I failed to establish with  https://github.com/carla-simulator/ros-bridge. So I just wonder if you could send me a example for the combination from carla UE4 to Rviz without .sh file, that would be more great for me to learn the combination.

Hello,everyone!
How to modify the name of the map coordinate (/map)?


Start using ark to handle binary installation

First suggestion for binary.rb with ark.
No chefspecs yet.

This look fine.  What else do you think this would need other than some specs?

Hmm.. I don't know. I think it's ok as it is but let me know if you have some ideas?

I added custom matcher for ark_put. Arks next release should include packaged custom matcher so it's only temporary.

Looks great.  I'd like to merge if you are ok with it.

Yeah, just go ahead


Restaurants-Self-Order - Week - 1

📝 Weekly Report For TNB Projects
Basic Details
🧪 Name of the Project: Restaurants-Self-Order
📅 Report Date: [02-20-2021]
📜 Team Work Report
✅ Accomplishments this week

Worked on "Staff portal"
Meetings about the item's DB design
Conducted a couple of interviews with possible backend designers, and hired 1 FE dev
On boarded the new FE dev
Reviewing the designs
And more...

🚩 Goals for next week

Get the FE dev to work on the staff portal
Finish the customer view design
integrate the BE with the staff portal

🔭 Long term goals

Finish both Staff portal and vendor portal FE & BE
Finish the mobile app customer view & vendor view

❗❓ Concerns
[ANSWER]
👨💻👩💻 Team Members Contribution Details
skybrandevs :

All previous corrections  and restrictions highlighted has been effected
Added the password reset and update components
Design components for SEE MORE section which contains USERS PROFILE, NOTIFICATIONS SETTINGS
YOUR FAVORITES and HELP.
User profile page update and view
Notifications page settings update
Notifications center, VIEW, DELETE AND MARK AS UNREAD
Part wire frame low fidelity design of tbnc account address input
location setup using Map
https://www.figma.com/file/edsNhzpjFn09dtO5AYQ5L8/food_new_version?node-id=0%3A1

emlex007 :

All previous corrections  and restrictions highlighted has been effected
Modifying all corrections on the app homepage
Enhance homepage font typefaces
Homepage clean up (fixed all white spaces)
Created Restaurant Menu meal pages
Added map to restaurant details page
Empty state for restaurant menu page
Bottom menu enhancement
https://www.figma.com/file/edsNhzpjFn09dtO5AYQ5L8/food_new_version?node-id=0%3A1

SazidAhmed with dhakal0kushal

Application for partner registration is implemented
application accept and receive functionality is implemented
display the applications in staff portal is implemented
Item Model is completed
Customization_Group Model is added
Customization_Items Model is added
Adding Item in Customization_Items is mapped with items and groups
Food Category has been updated
Bug in user's shop_branch has been fixed
Latitude, Longitude, Price range are added in shop branch
Dynamic Country model added for shop branch
https://github.com/Restaurants-Self-Order/Backend

yaya-BL

Meetings with backend team regarding the "Staff portal"
Meetings with backend team regarding the item's DB design
Conducted a couple of interviews with possible backend designers, and hired 1 FE dev
Meetings with the new FE dev to explain the project
Review the designs with giving comments
And more...

Designs are looking great 💯

@manishram Please note the update on this issue

@itsnikhil @yaya-BL forgot to put his own contribution to the project and hence he was not paid.
He updated his report with his past week's contribution, so please send him his weekly payment.
His account number can be found here: https://github.com/thenewboston-developers/Project-Proposals/blob/master/CSVs/project-teams.csv#L52


High cpu usage by Display module

I'm having a lot of fun with this, but ran into this performance issue.
I'm using the Display module to render a rectangular display, and getting 100% cpu usage from this line:
    /**
     * Timer tick: update dirty parts
     */
    _tick() {
        this._backend.schedule(this._tick);
        if (!this._dirty) {
            return;
        }

backend.schedule() is this line:
schedule(cb) { requestAnimationFrame(cb); }

It appears that, as long as there's nothing to do, it just executes and returns as fast requestAnimationaly possible. Adding a 2ms setTimeout to this._backend.schedule(this._tick) dropped chrome's cpu usage from 100% to 7%.
Implementing an optional rate limiter would trivial, but I wanted to make sure that I'm not using it wrong before submitting a PR?

Hi @Jon-Biz,
it is, generally speaking, browser's responsibility to throttle requestAnimationFrame freqency into suitable domain: https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame
As far as I now, the typical frequency is 60fps, which corresponds to ~16ms delay between individual calls. It is weird that a manual +2ms delay would drop the CPU usage as much.
Instead of introducing additional delays, we might simply stop the whole loop and wait until the nearest draw operation. This would remove unnecessary tick/schedule calls that are currently a no-op.
However, I am quite interested in how you got into this, because an empty requestAnimationFrame loop shall not be observable with respect to CPU usage. Please:

what is your CPU?
what OS/version?
what Browser/version?
are you actually drawing/animating during the loop, or is that a static display?
are you drawing ASCII/Unicode chars or Image Tiles?
can you add some instrumentation to measure+log the real delay between individual tick calls, so we can see the real frequency?

Thanks!


TN-2915 IRONMAN add Nigeria and 4 sites there (en_GB only)

https://jira.movember.com/browse/TN-2915

Sorry @ivan-c  I forgot to do this Friday. Can you please bring it live sometime before Tues AM?


Update enable-aad-authentication-azure-ssis-ir.md

Fix doc errors for preparing AAD auth with SQL MI.

@timchenyx : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

#sign-off

#sign-off

@chbain : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Added additional command tests

These tests are all mocked out, and do not execute the driver.
Removed the vagrant create driver since we test this already
in functional tests.


openapi3.ExtentionProps is unavailable in latest versions of kin-openapi

...\mod\github.com\meteran\gnext@v0.8.0\docs\structures.go:30:26: undefined: openapi3.ExtensionProps
...\mod\github.com\meteran\gnext@v0.8.0\docs\docs.go:46:5: unknown field ExtensionProps in struct literal of type openapi3.Info

openapi3.ExtensionProps should be replaced with map[string]interface{}
ExtensionProps: options.ExtensionProps should be replaced with Extensions: options.ExtensionProps
https://github.com/getkin/kin-openapi/blob/994d4f01c1e8dd613805668a7c10b568547f7789/README.md?plain=1#L279

temporary solution is to downgrade to kin-openapi v0.112.0

Hi, we're happy to see your interest in gnext 👋
Upgrading the kin-openapi library and fixing this issue has been submitted for review.

New release v0.9.0 has covered new version of kin-openapi.


dockerclient: detect ENOENT when copying

Detect cases where, when filtering the output of archiveFromFile() or archiveFromContainer() to net only sources which match a source path, we fail to match anything.  When that happens, return an os.ErrNotExist error from the archive's Close() method.  The archiveFromContainer() function now returns a ReadCloser where it previously returned only a Reader.
This aims to fix https://bugzilla.redhat.com/show_bug.cgi?id=1794768.

Note for reviewers: the dockerclient package isn't used in buildah; we have https://github.com/containers/buildah/pull/2111 to check for this there.

/test unit

/test unit

This should fix #137.

/lgtm


Fixed integer overflow issue in slice function for android

I am not a Java developer, but we were experiencing integer overflow issues while working with binary files. The issue was in the slice function. Start and end values were declared as int instead of long. Please review the code as there could be an unnecessary cast or something missed.

Thanks for the fix.
Looks in general fine but I assume you tested it for your case too

Thank you for merging, yes we've been using this as a patch on android for a month now, and we haven't spotted any issues.


How to close the port

In my page,i use
var SerialPort = require('serialport');
var port = new SerialPort('COM2', function (err) {
  if (err) {
    console.log('Error: ', err.message);
  }
  else{
	console.log('Success.');
  }
});

to new a serialport,when i load this page first time,It console.log ("Success").
But If refresh this page,It console.log( "Error").
And I know the reason is 'COM2' is not close.It issue fail when it new SerialPort().
And my question is:
how can i close the 'COM2' when the page is reload?
or has there a way to renew this 'COM2' to var port?

Thank You.

@reconbot  thank you your answer.
I know use port.close() to close the port.
and my problem is in my page I use
var port = new SerialPort()
When the page is load on first time, It can new a serialport to variable "port" success.
When we reload the page again, var port = new SerialPort() will be issue a Error,an at this case,when use port.close(); will issue an "close() function is not defined" error due to fail to New a serialport.
But I solve this case:
i use a global variable "port" to new the SerialPort, which variable "port" was not be initialized during the page reload.
In this case, when the page reload,the variable "port" still cantain the serialport Object.
So,port.close(),is done.
thank you again.


Update Item List

Some items are missing, e.g. rugs.
A more complete list can be found here: https://github.com/Cuyler36/MyHorizons/blob/master/MyHorizons/Resources/v1/Text/Items/ItemNames_en.txt

You can rename and reorganize the list I use with the program by simply editing the csv file, I am not interested in adding this list as it seems to just be item variants which you can already achieve with changing quantities.

It’s not the item variations I’m talking about. It’s that the list is missing rugs.


https://app.swaggerhub.com/apis/smartsdk/ngsi-tsdb/0.8.2 page does not exists

Describe the bug
the latest version of the API description is not available.
To Reproduce
Steps to reproduce the behavior:

visit page: https://app.swaggerhub.com/apis/smartsdk/ngsi-tsdb/0.8.2

Expected behavior
page exists and documentation is rendered and correct per new version
Environment (please complete the following information):
N/A

fixed


Make subprocess calls unicode-aware

Posted one comment, though it may apply to a few of the places where we access git commit IDs via subprocess.

Merged, thanks @sharmaeklavya2!


Add fasterrcnn example

Please check the tutorial.
If more details about Faster RCNN and its hyperparameters is required, please tell me I will add it.
Also if some theory about this model is required. Will add that as well.

Codecov Report

Merging #80 into master will decrease coverage by 0.57%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #80      +/-   ##
==========================================
- Coverage   56.00%   55.43%   -0.58%
==========================================
  Files          55       55
  Lines        1714     1692      -22
==========================================
- Hits          960      938      -22
  Misses        754      754

Flag
Coverage Δ

#unittests
55.43% <ø> (-0.58%)
:arrow_down:

Impacted Files
Coverage Δ

mantisshrimp/core/bbox.py
63.26% <0.00%> (-4.01%)
:arrow_down:

mantisshrimp/metrics/metric.py
53.33% <0.00%> (-2.92%)
:arrow_down:

...antisshrimp/metrics/coco_metric/reference_utils.py
19.28% <0.00%> (-2.00%)
:arrow_down:

mantisshrimp/metrics/coco_metric/coco_metric.py
37.83% <0.00%> (-1.64%)
:arrow_down:

mantisshrimp/models/mantis_rcnn/mantis_rcnn.py
37.50% <0.00%> (-1.10%)
:arrow_down:

mantisshrimp/core/mask.py
80.20% <0.00%> (-0.98%)
:arrow_down:

mantisshrimp/core/category.py
85.00% <0.00%> (-0.72%)
:arrow_down:

mantisshrimp/parsers/parser.py
93.75% <0.00%> (-0.19%)
:arrow_down:

mantisshrimp/parsers/splits.py
95.65% <0.00%> (-0.19%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66ebefa...b1eeb8f. Read the comment docs.

Use ReviewNB for checking the suggestions.

There is a lot of overlap between this tutorial and this one.
I do like the part while you over the different models we can use thought.
I have two suggestions:
1
Merge the two tutorials into one. But keep in mind that this is probably going to be the first tutorial beginners see, so try not to overwhelm them with information
2
Transform this into a advanced/extended tutorial, in this case you can even tell how stuff is happening in the inside, going really into detail, don't spare words.

I personally like more suggestion 2, which I think is going to be very complementary to the tutorials we already have and will facilitate a lot the lives of people wanting to get involved with the library.
But it's also a lot more work... well, your call at the end 😆

Personally Even I felt I should do 2nd thing. We can have 2 tutorials, first one as a QuickStart. And 2nd an in depth guide for this.

The code block does not look good on ReviewNB. But locally it is looking like a code block
like this
num_classes=None,
# transform parameters
min_size=800, max_size=1333,
image_mean=None, image_std=None,
# RPN parameters
rpn_anchor_generator=None, rpn_head=None,
rpn_pre_nms_top_n_train=2000, rpn_pre_nms_top_n_test=1000,
rpn_post_nms_top_n_train=2000, rpn_post_nms_top_n_test=1000,
rpn_nms_thresh=0.7,
rpn_fg_iou_thresh=0.7, rpn_bg_iou_thresh=0.3,
rpn_batch_size_per_image=256, rpn_positive_fraction=0.5,
# Box parameters
box_roi_pool=None, box_head=None, box_predictor=None,
box_score_thresh=0.05, box_nms_thresh=0.5, box_detections_per_img=100,
box_fg_iou_thresh=0.5, box_bg_iou_thresh=0.5,
box_batch_size_per_image=512, box_positive_fraction=0.25,
bbox_reg_weights=None

@ai-fast-track can we get your input?
I feel like there is a lot of overlaps in the tutorials right now, between this one, wheat and parser.
Should we avoid overlap or is it a good thing to have multiple tutorials with different levels of details?

Helps issue #39

@ai-fast-track can you please have a look ?

@lgvaz @oke-aditya, I think we should add this notebook even if there is some overlap with wheat.ipynb. The latter can be used as quick showcase and highlighted (expose source code) in the README

@ai-fast-track Suggestions are made. Can you please have another look?


See issue https://github.com/KhronosGroup/SPIRV-Tools/issues/3159 for more details.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


remove duplicate packages

Solution for #743

Current coverage is 11.15% (diff: 100%)

No coverage report found for development at e3cf774.

Powered by Codecov. Last update e3cf774...be77b58

Thank you.


feature request: set offset and scale

I need to adjust the position and scale of the Zoomify layer in order to calibrate it and be able to use geo coordinates on it. I couldn't find an option to set offset and scale for this in the plugin or in Leaflet.
I saw it was already added to this fork of the plugin, which is unfortunately not updated to Leaflet 1.0.
As I'm not even sure if the plugin is the component that should solve it, I also cross posted here asking for more general advice and other strategies.
Thanks!

Not sure if that will be the task of this plugin. Do you only want to display your own zoomify image or project it over another tile layer? If it is the former, you can use a custom CRS to accomplish custom scaling and offset. See some other work of mine: https://github.com/cmulders/cask_virtualmicroscopy/blob/gh-pages/annotated_zf/index.html or live version: http://anatomie01.lumc.nl/microscopy/zf/?zoom=0&lat=-0.000310&lng=0.000750
In particular this code:
//Slide parameters, res in m/pixel
	var slide = {id:'zebravis', width:921600, height:380928, res:1.627e-9};

//Custom CRS to translate between pixels and meters.
    L.CRS.Slide = L.extend({}, L.CRS.Simple,{
        transformation: (function(){
            var longestSide = Math.max(slide.width, slide.height);
            var maxZoom = Math.ceil(Math.log(longestSide / 256) / Math.log(2));
            var correctedRes = slide.res*Math.pow(2, maxZoom);
            return new L.Transformation(1 / correctedRes, 0, -1 / correctedRes, 0)
        })()
    });

Check the documentation for custom crs for the details.
Note: this code was for leaflet-1.0beta

Thanks a lot!
It's only going to be one Zoomify image and overlays (markers and paths), but no other base layer.
The CRS and your example is really helpful, thanks! I've got two questions though:

How do you calculate the res:1.627e-9? Is this a pixels/meter ratio?
What type of coordinates are you using in annotations.js? Are these geo coordinates, pixel coordinates or even something else?

Good luck! If you want any help, please let me know. Was quite a challenge to figure it all out the first time. But if you get the calculations right, it works like a charm.
Let me know if the plugin itself does not behave like expected :).

Thanks a lot!
Your advice would be highly appreciated.

Using your code and changing the dimensions and resolution of the slide/map layer will probably not be enough. Your code adjusts the zoom, but how do I translate along lat/long?
Do I have to change the projection from L.Projection.LonLat (being used by L.CRS.Simple) to L.Projection.SphericalMercator being used by my source or should I even use and extend L.CRS.EPSG3857 instead of  L.CRS.Simple?

Thanks again!

After some thought, the problem is not really trivial. The issue is that the Zoomify plugin now loads the map at '0.0'. I thought that using a offset in the CRS transformation to a custom origin it could be done. However, the tile layer starts to load those tiles instead (e.g. requesting not Tile 0,0 but 14,15). Therefore, the (Zoomify) plugin or leaflet itself needs to correct for that.
Not only that, if your origin is not on a tile boundary, then your tiles also need to be shifted again to accommodate the misalignment.
The image I showed, is a independent image without any other reference, therefore the 0,0 origin was fine for me.
Maybe also ask in https://gitter.im/Leaflet/Leaflet !

Thanks. After all, I went on a totally different route without Zoomify, which I think makes more sense in my case.
I used QGIS to geo-reference my custom layer, exported it as GeoTIFF, uploaded it to Mapbox and baked it into a map style while mixing it with additional map features from Mapbox/OSM.

Ah nice, probably a much better solution!


bd_rate_report.py returns NaNs in JSON

These aren't technically allowed. Consider returning string "NaN" instead.

There is currently string substitution to fix this on the frontend for now.


[DESIGN] Review / Revise results text to be more explicit

Amber has suggested we review the results text. Here are a few examples to get the thought process going. We can chat at next D&T meeting.
ie, on the homepage if an entry type tab is selected (the All tab can still say "results"):
40 Cases of 50 (page 1 of 3)
5 Collections of 5 (page 1 of 1)
ie: on the collections landing page:
all: 5 Entries of 5 in this Collection / in the Government of Canada collection
cases tab: 5 Cases of 5 in this Collection / in the Government of Canada collection

Based on usersnap feedback - This should also be designed alongside displaying selected search filters (in addition to the red number that shows up on the filters button) * I have a few mockups for this


Memory Leak for macOS

in DDLog.m, there are some code for macos:
- (id)init {
    self = [super init];

    if (self) {
        self._loggers = [[NSMutableArray alloc] initWithCapacity:4];

#if TARGET_OS_IOS
        NSString *notificationName = @"UIApplicationWillTerminateNotification";
#else
        NSString *notificationName = nil;

        // On Command Line Tool apps AppKit may not be avaliable
#ifdef NSAppKitVersionNumber10_0

        if (NSApp) {
            notificationName = @"NSApplicationWillTerminateNotification";
        }

#endif

        if (!notificationName) {
            // If there is no NSApp -> we are running Command Line Tool app.
            // In this case terminate notification wouldn't be fired, so we use workaround.

            atexit_b (^{
                [self applicationWillTerminate:nil];
            });
        }

#endif /* if TARGET_OS_IOS */

There is a workaround:
            atexit_b (^{
                [self applicationWillTerminate:nil];
            });

It will cause DDLog will be retained and never be released. Right?

While I'm not 100% sure if those two places really leak, I've also captured self weakly there now.
Those dispatch sources are removed when the log file is rolled, so in theory it might not completely leak. But better safe than sorry.
Thanks for reporting these!

In my app, the DDFileLogger will still live when I close the window ( the window use a logger, many windows will use many loggers). I use the Instrument to debug it and find the logger be captured.


Fix all function for torch array when axis is None

dim=None results in an error, because torch uses a different function when dim is not given.
I would like to add a test, but it seems like one has to to adapt the tests that they also run for torch arrays and that is a bit too much for me now.

:books: Documentation preview :books:: https://equistore--269.org.readthedocs.build/en/269/

Okay added now a new test file that will use the torch version if torch is available. I am not sure if we have at the moment a github action running with torch, but at least this way, we have the fix covered in the future when we add such a github action.

Yes, we run the operations tests with torch installed!


2 different syntaxes for json mapping

I see both Json mapping syntaxes:
.create table TestTable ingestion json mapping 'TestMapping' '[{"column":"TimeStamp","path":"$.timeStamp","datatype":"datetime"},{"column":"Name","path":"$.name","datatype":"string"},{"column":"Metric","path":"$.metric","datatype":"int"},{"column":"Source","path":"$.source","datatype":"string"}]'

and
.create table MyTable ingestion json mapping "JsonMapping1"
'['
'    { "column" : "messageid", "datatype" : "string", "Properties":{"Path":"$.message-id"}},'
'    { "column" : "userid", "Properties":{"Path":"$.user-id"}},'
'    { "column" : "other", "Properties":{"Path":"$.other"}}'
']'

Is it so that the first one is deprecated, and should be changed to the second one?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d5808ed2-5d09-4c56-dc0b-4e3650f8fc16
Version Independent ID: 10ebaf9d-0cc0-b59b-ae95-63bc7b1be4b5
Content: Ingest data from Event Hub into Azure Data Explorer
Content Source: articles/data-explorer/ingest-data-event-hub.md
Service: data-explorer
GitHub Login: @orspod
Microsoft Alias: orspodek

@deyanp Thank you for bringing this to our attention. We are investigating and will update the documentation as appropriate.

@deyanp They are both the same. The first example is in-line and the second is extended to show each column definition per line. Please let me know if you have any additional questions, as I am going to  close this thread but, if there are further questions regarding this matter please comment and I will gladly continue the discussion.

The 2nd example wraps path in Properties, 1st one not, but nevermind ...

@deyanp I see what you are referring to. I have assigned this to the content team to evaluate and update as a doc-enhancement.

@deyanp thank you for your feedback. The first example only performs ingestion mapping. The second example adds the event system properties mapping. It is only needed if you selected "Event system properties" in the "Data Connection" pane. Hope this is clear.

@orspod , @Mike-Ubezzi-MSFT , I have the feeling I am being actively discouraged to point out discrepancies in the docs. Compare the 2 lines below, each one extracted from the 2 mappings at the beginning of this thread, and tell me what is the difference (besides column names and json paths):
{"column":"TimeStamp","path":"$.timeStamp","datatype":"datetime"}
{ "column" : "messageid", "datatype" : "string", "Properties":{"Path":"$.message-id"}}

Then check your own docs where it says:

Deprecated mapping format The previous version's mapping format allowed a different definition for each mapping, where properties could be specified without the Properties dictionary. The deprecated format is shown below.

Happy to hear that I have misunderstood it all, but if not - you'd better correct your docs, as it misled me in creating table mapping using a deprecated format (quick starts are the thing which people usually follow) ..

@deyanp thanks for your feedback and for reaching out again with clarification on your question. I now understand your confusion. I am adding @kerend the dev on this feature to assist in responding to your question. We will also make sure to update the docs accordingly to prevent future issues.

@kerend There is still one remaining question on this subject, which I posted to the ADX Teams channel but will posy here as it is a good discussion item and there is no ADX UVOC forum for feature requests:
"I will assume that there is no other way of transposing child array into columns without pre-processing the json, flattening it and after that re-publishing it to another event hub for consumption by Azure Data Explorer." <- Is there an Event Hub option to flatten or elevate child properties in JSON so that the column mapping from JSON child property is direct? No prepossessing required?

@deyanp I received some feedback from the PG. The process will be to ingest the JSON as is, and then use the Update Policy to transform the data from a source table to a target table using the ADX mv-expand operator.

We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.


Bytes type

Main question:  doesn't this add 1-2x datum copies, versus the current &[u8]?
Many API functions are looking for a RoByteRef (read-only byte reference), yes?  Is it possible to maintain that efficiency while also having this added flexibility, "having our cake and eating it too"?

For some types, yes, but mostly no. Currently only converting &[u8] and usize causes a copy. I can't think of any situation where a large amount of &[u8] data will get stored, we usually have Vec<u8>.  I suppose we could have an accidental insert(&large_vec_key, &large_vec_value) that would duplicate the Vec. For small types, there's always the option to use Vec, but the the conversion would be at the call site instead.
I guess we could remove the automatic conversions for &[u8] to avoid the above accidental copies. It will rarely be necessary, other than for byte literals such as b"probably only used in examples". But the the examples would look more complicated: insert(b"key".to_vec(), b"value.to_vec()) instead of insert(b"key, b"value").

It makes handling mmap'd data less efficient.  Merged anyway; let's see how it goes.


Qubit remapping bug in qiskit 0.5.[0-2]

Qubit remapping appears incorrect in qiskit-ionq 0.5.* we are preparing a hotfix and working to mitigate the impact of this, as it leads to apparently incorrect results, as mentioned in https://github.com/qiskit-community/qiskit-ionq/pull/154#issuecomment-2259834170

We've yanked versions 0.5.0, 0.5.1, 0.5.2 in light of this bug, results that appeared incorrect in previous versions of qiskit-ionq should now be retrievable correctly in qiskit-ionq 0.5.3 (no data loss has occurred).


JSON output of tx spend includes "hash", which will change when signed

The txhash only means something when the transaction is signed.
shinichi@ilya:~/source/go/src/github.com/aeternity/aepp-sdk-go[climocknetwork*] aecli-py -u http://localhost:3013 tx spend $SENDER $RECP 10 --json
{
  "data": {
    "tag": 12,
    "vsn": 1,
    "recipient_id": "ak_Egp9yVdpxmvAfQ7vsXGvpnyfNq71msbdUpkMNYGTeTe8kPL3v",
    "amount": 10,
    "fee":<PHONE_NUMBER>0000,
    "sender_id": "ak_2a1j2Mk9YSmC1gioUq4PWRm3bsv887MbuRVwyv4KaUGoR1eiKi",
    "payload": "",
    "ttl": 0,
    "nonce": 0
  },
  "tx": "tx_+FEMAaEBzqet5HDJ+Z2dTkAIgKhvHUm7REti8Rqeu2S7z+tz/vOhAR8To7CL8AFABmKmi2nYdfeAPOxMCGR/btXYTHiXvVCjCoYPMEWRWAAAAIBowtv6",
  "hash": "th_2BKJNpj4TWt133JNmoKXASFiXN9wr99qgaXPRGkwpsAdrAtYwy"
}
shinichi@ilya:~/source/go/src/github.com/aeternity/aepp-sdk-go[climocknetwork*] aecli-js -u http://localhost:3013 tx spend $SENDER $RECP 10 --json
error: missing required argument `nonce'
shinichi@ilya:~/source/go/src/github.com/aeternity/aepp-sdk-go[climocknetwork*] aecli-js -u http://localhost:3013 tx spend $SENDER $RECP 10 10 --json
{"tx":"tx_+FEMAaEBzqet5HDJ+Z2dTkAIgKhvHUm7REti8Rqeu2S7z+tz/vOhAR8To7CL8AFABmKmi2nYdfeAPOxMCGR/btXYTHiXvVCjCoYPJvVhyAAACoBYVU9d","params":{"tag":"12","VSN":"1","senderId":"ak_2a1j2Mk9YSmC1gioUq4PWRm3bsv887MbuRVwyv4KaUGoR1eiKi","recipientId":"ak_Egp9yVdpxmvAfQ7vsXGvpnyfNq71msbdUpkMNYGTeTe8kPL3v","amount":"10","fee":"16660000000000","ttl":"0","nonce":"10","payload":""}}

technically the hash of a transaction is the base58 encoded black2b hash of the rlp encoded data of a transaction, and therefore valid for any type of transaction.
Whenever the hash is going appear in the blockchain because it is of a signed transaction it doesn't preclude the idea of generating hashes for any transaction


Make template filepaths respect text case

This makes the app work on OSes that are case-sensitive, such as GNU/Linux.

1nce again, Tip! Thanks :)


Now deployment

Deployment with Zeit's Now

Partially blocked by https://github.com/gothinkster/realworld/issues/42


Data Picker doesn't reposition when it resizes – the hydrogen-may project has some sample data which makes the data picker grow larger, and then it ends off the screen

fixed as the data picker no longer resizes


Tensorial product not displaying (in localized versions)

Hi team,
I moved this from here: https://github.com/MicrosoftDocs/quantum-docs-pr.it-IT/issues/3
The issue replicates in the localized versions.
After further investigation, we concluded this is due to a source issue. It has been escalated in Loc Issue #240894

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: feb830cc-7d9b-c13e-4652-115352ea1e29
Version Independent ID: f9484b57-1dc8-06a9-ac6d-cdc7c4769981
Content: Linear algebra for quantum computing - Microsoft Quantum
Content Source: articles/overview/algebra-for-quantum-computing.md
Product: quantum
GitHub Login: @Bradben
Microsoft Alias: bradben

Fixed with LaTeX localization issue #860

Fixed with LaTeX localization issue #860


Internal representation

As distinct from the the TOML format, it might be worth rethinking how we represent parameters internally. There are pros and cons to each but it is worth explaining them a bit.  As far as I can see, there are 3 broad options:
1. A package of constants
Conceptually this is the simplest: we get rid of the notion of a "parameter set" completely, and you simply have a package with a bunch of constants:
module CLIMAParameters
const param1 = 1.2
...

(note: this code would be generated by the toml file)
This is very easy for the user to make use of:
import CLIMAParameters: param1

f(x) = x*param1

The original reason we didn't use this is that well, they're constant: you can't modify them. However now with the new Preferences system, we would change the definition above to
module CLIMAParameters
using Preferences
const param1 = @load_preference("param1", 1.2)
...

and then the user could modify it by the LocalPreferences.toml file in their current experiment directory
[CLIMAParameters]
param1 = 1.3

This would then correctly trigger recompilation when the value is modified, and the changes would only apply to the current experiment.
However there are a couple of downsides:

every new set of parameter values will trigger a round of compilation, e.g. when doing UQ, every single run will require a different set of precompilation caches. This could cause significant technical issues here (e.g. by default, Julia keeps at most 10 precompilation caches of each package).
users can't add new parameters for a given experiment
there can only exist 1 value of the parameter in the current process: this means that each experiment must be run standalone. Obviously this won't be a problem for most model runs, but there may be cases where you want to run say a bunch of column models with different parameter values.

We might be able to work around these by defining them as Refs, but then user code would look like
f(x) = x*param1[]

which isn't particularly nice.
2. Constant functions
This is the approach we use now: a function which returns a constant value:
module CLIMAParameters
param1(::ParameterSet) = 1.2
...

At the user level, this involves:

passing around a "parameter set" object to the different functions
evaluating the function in the scope in which you require the parameter. Since the usual symbol name is already taken by the function, this has lead to an ugly and somewhat confusing convention of using underscores:

function f(paramset, x)
   _param1 = param1(paramset)
   x*_param1
end

In cases where you might be doing multiple runs with different values of a given parameter, the intention was that you would create a new parameter set type with fields for the modifiable parameter values:
struct CustomParameterSet
   param1::Float64
end
param1(ps::CustomParameterSet) = ps.param1

This is roughly what @charleskawczynski did in TurbulenceConvection.jl: https://github.com/CliMA/TurbulenceConvection.jl/blob/2a0cea8c184b5f8ffe13f9862aaf3197aa8eca56/driver/parameter_set.jl#L43
As a possible fix to the interface issue, in #57 I proposed a slight modification.

a parameter is now a singleton type (i.e. a struct with no fields): the name of this type should be descriptive
its value can be extracted by a value function
we use getproperty overloading of the parameter sets to lookup by symbol.

Roughly, the (generated) module now becomes
module CLIMAParameters
struct DefaultParameterSet end
struct Parameter1 end
value(::DefaultParameterSet, ::Parameter1) = 1.2

@inline function Base.getproperty(ps::DefaultParameterSet, name::Symbol)
    if name == :param1
        value(ps, Parameter1())
...

and then the user code looks like:
f(ps, x) = x*ps.param1
# or you can define it as a local variable
function f(ps, x)
   param1 = ps.param1
   # or
   @unpack param1 = ps
   # or
   (;param1) = ps
   # or the long version
   param1 = CLIMAParameters.value(ps, CLIMAParameters.Parameter1())
   x * param1
end

After some consideration, I think something along these lines would be my personal preference.
3. A dictionary
This is at the other end of the extreme: basically CLIMAParameters just gives a dictionary of values, and it is up to the model components to extract the necessary values at construction (to avoid the cost incurred of looking up dictionaries), e.g.
struct ComponentA
   param1::Float64
   ...
end
ComponentA(ps::ParameterSet) = ComponentA(ps[:param1], ...) # values are copied at construction

f(c::ComponentA, x) = x * c.param1

This would mean that components could be run completely independent of CLIMAParameters. However it could mean that we end up duplicating a lot of code and constants and end up just pushing the problem down into individual components, e.g. Thermodynamics.jl would probably end up defining its own ThermodynamicsParameters object to capture all the parameters needed for thermodynamics.

We definitely do not want to trigger a recompile for every parameter change. Once you have a full ESM, this will become unwieldy. We want parameters to be runtime constants of some form.

In 3. The "extraction" can be automated by tagging parameters? Then do we still need duplicate code, or can we not write the boilerplate in ClimaParameters? e.g. ClimaParameters.create_parameter_struct(component, tags)
You are right, this leads to many structs, one for "component" which contains only the parameters used within this components source code. But once again, are we stll not talking (ignoring NN parameters) at ten's of floats per component and maybe 10-20 structs?

In 3. The "extraction" can be automated by tagging parameters? Then do we still need duplicate code, or can we not write the boilerplate in ClimaParameters?

We could probably figure something out, e.g. a macro
@parameterstruct(paramdict, ComponentX)

which expands to
struct ComponentXParameters
   param1::Float64 # the set of parameters tagged "Component1"
   ...
end

A couple of things worth noting:

how specific would the tags be? e.g. it doesn't make sense to pass a struct containing all the EDMF parameters to say a simple Smagorinsky model.
I would have to think carefully about how experiment-specific parameters would work. e.g. if  someone is doing an experiment on TurbulenceClosures.jl, and wants to define some new parameters for a specific EDMF formulation, then the struct definition can't live inside CLIMAParameters.

RE the tag specificity. I think this is why I keep mentioning about the organization of code, for us what do we mean by component (does this relate to a git repo?) or subcomponent (does this relate to a module?) etc.
The tag specifics are a trade-off. broad tags (e.g. a git repo)  will produce fewer, larger structs, narrow tags (e.g. a module) will produce more, smaller structs.

Modules are namespaces: i.e. a way to encapsulate things so that names are unique within a module. Packages are modules with a bit of extra metadata to resolve dependencies (and typically match 1-1 with a repo).

After some further discussion with @odunbar, we agreed on the following approach (based on option 3, and similar to what is done in OceanTurbulenceParameterEstimation.jl:

A parameters set is basically a dictionary, which contains the values and associated metadata.

Model components which use any parameters would need to access them when they are constructed (i.e. not while the model is running), and load them into their own struct (or similar object), or subcomponens: e.g. suppose we define a model such as:

struct AtmosModel
   # parameters required directly by AtmosModel
   gravitational_acceleration
   coriolis
   ...
   # subcomponents
   thermodynamics
   ...
end

Then the constructor
AtmosModel(paramset; thermodynamics=MoistEquilibrium())

would do something like
# extract parameters required by model
gravitational_acceleration = get_parameter(paramset, :gravitational_acceleration, "AtmosModel")
coriolis = get_parameter(paramset, :coriolis, "AtmosModel")
# recursively extract subcomponents
thermodynamics = update(MoistEquilibrium(), paramset) # constructs a new MoistEquilibrium object with the relevant parameters

Once the components are constructed, they would refer to their own object to get the parameter value (i.e. we shouldn't be looking up parameter sets once the model is running). For example, Thermodynamics.gas_constant_air:

https://github.com/CliMA/Thermodynamics.jl/blob/31231f74fd46268f9f6231a5f24aa7cfcdb000a2/src/relations.jl#L72-L77
would now refer to the specific MoistEquilibrium object instead of the parameter set:
function gas_constant_air(thermo::MoistEquilibrium, q::PhasePartition{FT}) where {FT}
    return thermo.R_d *
           (1 + (thermo.molmass_ratio - 1) * q.tot - thermo.molmass_ratio * (q.liq + q.ice))
end

We can build some convenience functions for automatically mapping parameters to struct members; also this would allow us to log exactly which parameters were used in a given experiment, and by which components, e.g. by defining

function get_parameter(paramset, name, component)
    # log that the parameter was used
    @info(paramset, name, component)
    # get the value
    return paramset.dict[name]
end

This is significant change from our current approach: unfortunately, there isn't an easy way to gradually switch over, but it is probably worth prototyping this in a single package (e.g. TurbulenceConvection or Thermodynamics?).

I love this.

I don't mean to open a can of worms, but @kpamnany and I have already voiced concerns about the distributed approach (option 3), as it does not guarantee the "single parameter" requirement that we were given in the very early stages of developing a design solution to CliMA's parameters.
If we go down this road of distributed parameters, then it sounds like we've relaxed this requirement. The existing approach (using methods, and not types) ensures this by construction. The distributed approach, where each package / module has its own parameters via separate types, is subject to having different parameters and the responsibility of ensuring a single value is used for all parameters is pushed to the outer-most layer (i.g., tests/experiments/drivers for all other repos, i.e, the user). Here's a simple example:
atmos = AtmosModel(
thermo = ThermoParams(; grav = 1),          # override for grav in Thermodynamics.jl
microphys = MicrophysicsParams(; grav = 2), # override for grav in CloudMicrophysics.jl
)

Arguably, this is something that obviously looks wrong and "won't typically happen", but the point is that it can happen. Let's consider a slightly more complex situation:
atmos = AtmosModel(
thermo = ThermoParams(),                    # I can use Thermo's defaults, right? ¯\_(ツ)_/¯
microphys = MicrophysicsParams(; grav = 2), # override for grav in CloudMicrophysics.jl
)

And now we have inconsistent parameters.
Now, I'm not saying that we shouldn't take the distributed approach--our original design choice was sketched out based on the given requirements at the time. This distributed approach may look fine / easy / sensible when looking at a single package, like Thermodynamics or Oceananigans. But, our existing method-based design satisfies the single parameter requirement across all packages.
My preference would be to open two issues: 1) regarding the hard-aches with our existing design and 2) regarding additional needs from our existing design. For 2), #58 looks great, and that can work with our existing design. For 1), I imagine that we will be able to alleviate hard-aches with our existing design if they are laid out. I think I recall a preference for α = param_set.α over α = CP.α(param_set), but I don't see why this is problematic and I've yet to see a concrete technical reason to change our existing design.

Perhaps it would be good to start by documenting the requirements of the parameters feature?

Some distinction here may help w.r.t. to when we do and don't need consistent parameters:

All "planet" parameters should be the same across model components. This includes thermodynamic constants, planetary rotation rate, mean radius etc., and crucially, e.g., the latent heat of vaporization at the triple point (land, ocean, atmosphere, ice should use the same to ensure energy conservation for the coupled model). These should all live in some "planet" file shared by all model components.

Many other parameters can plausibly be different for different model components. For example, the Smagorinsky coefficient for an atmosphere and ocean model may well be different. We do not need to enforce consistency here.

Many other parameters can plausibly be different for different model components. For example, the Smagorinsky coefficient for an atmosphere and ocean model may well be different. We do not need to enforce consistency here.

Wouldn't it still be safer in that case to have Smagorinsky_ocean and Smagorinsky_atmos coefficient, so that we keep the names unique across the codebase?

I think a good usage example was when for debugging purposes we wanted to set the freezing temperature to something very low to test the TurbulenceConvection.jl package. This required overwriting T_freeze for Thermodynamics.jl, CloudMicrophysics.jl and TurbulenceConvection.jl at the same time. It would be nice to still be able to do this easily and in one place, without having to check on individual packages.

3. Many other parameters can plausibly be different for different model components. For example, the Smagorinsky coefficient for an atmosphere and ocean model may well be different. We do not need to enforce consistency here.

Wouldn't it still be safer in that case to have Smagorinsky_ocean and Smagorinsky_atmos coefficient, so that we keep the names unique across the codebase?

Maybe. But if (one day) atmosphere and ocean use the same CFD code base, you couldn't easily do that. The broader point is: there are parameters that are not "universal" constants, and at least for experimentation, there may be situation where we want them to be different in different model components.

Sorry I just saw this thread, and thanks for voicing these concerns, @charleskawczynski ! I think to summarize, what our current setup does is remove the word "enforce" in the requirements and replace it with "make it difficult to change". As @kpamnany  and @tapios note,I think this results in a softening of the need for universal parameters; however in practice, it seems to be the use case that we would rather have such common sense principles over enforcement, as it is likely we will require such flexibility down the road.
On @trontrytel  and @tapios conversation. Yet another option is (if relationships are known between constants)

you define smagorinsky only in the parameter file with a description "atmospheric smagorinsky"
this gets distributed around to all components that want it.
when you build ocean=OceanComponent you create a "derived parameter" smargorinsky  = smagorinsky + epsilon
all ocean code will call ocean.smagorinsky

The difference here to previous suggestions is that only one constant is defined in the parameter file, and, changes to that parameter occur at the component level of the use case (and no further). You can also protect from confusion by renaming the local smagorinsky (e.g. ocean_smagorinsky to indicate it is different). If not then you know where the parameter is defined because it is called as ocean.parameter and so you know it is defined in the OceanComponent constructor.

Rather than building support for "parameter sets" into the model constructors, it might be less intrusive to require individual components (eg mutable struct AtmosModel to define a function
set!(model, parameter_set)

We have prototype support for this functionality in Oceananigans, and it's useful for running a few different models without recompiling or reallocating memory for a new simulation.
The main benefit of this pattern is that it simplifies the model constructors, which seem to end up being complicated pieces of code.

Given the discussion here:
https://github.com/CliMA/TurbulenceConvection.jl/pull/766#discussion_r795078598
Options 1 and 2 aren't really going to be feasible if you get into arrays of hundreds or thousands values.


Ansible 101 Live Stream Episode 8 Agenda - May 13

Previous episode agenda: #257
Video link: https://www.youtube.com/watch?v=CYghlf-6Opc

[x] Thank you SO MUCH to all the subscribers—10K!
[x] Thank you SO MUCH to GitHub Sponsors and Patreon Patrons:

GitHub: (one very generous private sponsor), @OpsTalkJordan, @captainwasabi, @vijaycs85, @danigu, @woodrad, @vitabaks, @apropox, @dragos-dumi, @aidanpl
Patreon: Coding Paradigms (HUGE thanks!), Friederich K.

[x] Check out Raspberry Pi Cluster, Episode 2 - Setting up the Cluster
[x]  Questions and notes from last week's episode:

J Mane asked: "Is it good practice to add molecule test in pipeline.....ie first run playbook on docker , if success then run it on actual server."
Baptiste Mille-Mathias: "it's good also to have a yaml validation inline in your $EDITOR."
Filippo Projetto: "Do you lint playbooks manually or use a tool to make sure that at least the styling issues are auto fixed?" (best is pre-commit hook).
Jordan Peterson: "Do you name includes and imports? That's usually the only time I don't name my tasks." (sometimes yes, mostly no)
Mike Sawyers "you could also use molecule login" (this is why I love livestreams!)
Mohammad Nur Aabideen: "Thanks! did you cut your own hair?" 💇♂

[x] New book version (1.23) is out today!
[x] Testing Ansible Playbooks with Molecule (chapter 12, formerly known as 11; see #101 and #264)

Next episode: https://github.com/geerlingguy/ansible-for-devops/issues/269


chore(package): Improve package.json

:tada: This PR is included in version 1.2.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Consistent aws timeouts

Timeouts are now set for all HTTP clients. Also workaround nasty Viper bugs.

Review please @tusharm, @pavelnikolov, @jtblin.


Task: Forward port client authentication & others

Description
Porting forward the client authentication support from https://github.com/PocketRelay/Client/issues/16 and other features that have had more attention in the standalone client version
Checklist

[ ] Client authentication is supported
[ ] Any other additional changes or support is pulled over from the standalone version

Closing, implemented in v0.0.4


Update GOVERNANCE.md

hello this is my comment.

is the goal of this PR to document the processes we have now?

is the goal of this PR to document the processes we have now?

I think it was mostly a "bro, can we even PR?" experiment, which I think was successful.
Closing in favor of #21 (I added a suggested change there)


Add Amazon readme

Adding Amazon to the friends. from https://twitter.com/lorenc_dan/status/1483576185196093441?s=20

Thanks @rothgar , just a quick Q , are you representing amazon (employed there)?


pepper: spin polarization

Need to incorporate into pepper and probably chili a better structure to handle spin polarization from various mechanisms.

The last thing to add would be a built-in mechanism for a recombination triplet. It is not obvious to me how we could do that in a general sense.


Potential null pointer dereferences after calling raxStackPop()

This is just one example that I'm not sure I fixed correctly (and that's why it's not a PR yet).
diff --git a/rax.c b/rax.c
index 7dcf045..9a8fcda 100644
--- a/rax.c
+++ b/rax.c
@@ -1056,7 +1056,7 @@ int raxRemove(rax *rax, unsigned char *s, size_t len, void **old) {
             h = raxStackPop(&ts);
              /* If this node has more then one child, or actually holds
               * a key, stop here. */
-            if (h->iskey || (!h->iscompr && h->size != 1)) break;
+            if (h != NULL && (h->iskey || (!h->iscompr && h->size != 1))) break;
         }
         if (child) {
             debugf("Unlinking child %p from parent %p\n",

Other occurrences appear in:
10. rax.c:1468: error: NULL_DEREFERENCE
      pointer `it->node` last assigned on line 1461 could be null and is dereferenced at line 1468, column 21.

11. rax.c:1608: error: NULL_DEREFERENCE
      pointer `it->node` last assigned on line 1517 could be null and is dereferenced at line 1608, column 26.

12. rax.c:1665: error: NULL_DEREFERENCE
      pointer `it->node` last assigned on line 1517 could be null and is dereferenced at line 1665, column 17.

13. rax.c:1755: error: NULL_DEREFERENCE
      pointer `n` last assigned on line 1754 could be null and is dereferenced at line 1755, column 25.

These may well be false positives found by infer or simply a rare edge case. If it's the latter, we should address it somehow.

You can ignore the rest of the errors if you run infer yourself. One of them is a false positive (see https://github.com/facebook/infer/issues/1317) and the rest occur in the test suite.

i see the loop in the patch you provided runs until it reaches h != rax->head
so i don't see how it's possible that raxStackPop will return NULL.
anything i'm missing?

As I said, I might be missing something which is why this isn't a PR instead of an issue.


Ability to assign non-weapon model upgrades to models

For example: a Daemons squad can take an icon and an instrument, each of which goes on a specific model. WIBNI these could be assigned, just like weapons, thereby allowing a different TTS model to be selected for these specially-equipped models?
I think this might be possible because although these are obviously <selection type="upgrade">, they do also have number="1".

The number attribute looks like a good hook for a special case along the same lines as the other upgrade special cases so long as it doesn't generate false positives.
https://github.com/ThePants999/Yellowscribe/blob/main/bin/Unit.js#L262
I think a good place to start would be collecting as many non-weapon model upgrades as possible into a single roster and testing that they share the number=1 attribute, then collecting some representative selection of model upgrades that shouldn't be assigned and ensuring that they don't have the number=1 attribute. If those assumptions hold true, then we can create a special case in Unit.js, check the output, and write a test for the new functionality.
Could you explain a bit more about how the testing system works?
https://github.com/ThePants999/Yellowscribe/blob/main/tests/roster.test.js
It looks like in order to get a human-readable data format you'd use the roszParse function and then write the output to a txt file. Are there existing utility functions for this?

The test framework is very simple. You add a roster file to samples/, and add a test case to roster.test.js referencing it. Then run the tests; your new test will fail but will generate a text file under tests/ showing the parsed output generated from that roster. Fix code and re-run until that parsed output is correct, then rename the file to <testcase-name>.approved.txt, and from then the test will pass, but will start failing if the code is subsequently changed such that that roster no longer generates exactly that output.