Important Notice: Deprecation of Connected Vehicle API Version 1 and Migration to Version 2

Hi, for your information
I received the following email from Open Innovation Portal<EMAIL_ADDRESS>Upgrade now for improved functionality and performance with Version 2 of our Connected Vehicle API.

*Important Notice: Deprecation of Connected Vehicle API Version 1 and Migration notice to* Connected Vehicle API Version 2

Dear Developers,

We want to inform you that the Connected Vehicle API Version 1 will be deprecated and removed from our portal on April 30, 2024.

As you may know, we're continuously striving to improve our services and enhance your experience with our developer portal. In line with this commitment, we are excited to announce the availability of the latest version, Version 2, of our Connected Vehicle API.

Please transition to Version 2 for improved functionality and performance. Explore the latest features here ( https://developer.volvocars.com/apis/connected-vehicle/v2/overview/ ).

Connected Vehicle API V2.0 ( https://developer.volvocars.com/apis/connected-vehicle/v2/overview/ )

If you need assistance, our support team is here to help - please feel free to get in touch with us at [developer.portal@volvocars.com](mailto:developer.portal@volvocars.com).

Thank you for your cooperation.

Best regards,

Volvo Cars Developer Portal

Yeah, thanks for this information.
We are completely on V2 right now!

Any idea why i don't see most of the sensors then for the last few days? Do we need to alter the configuration? Get a new key? Anything like that? Thanks
That's all I get at the moment:
}
Feb 19 15:27:41 volvo2mqtt [107] - DEBUG: Response status code: 200
Feb 19 15:27:41 volvo2mqtt [107] - DEBUG: {
  "data" : {
    "serviceWarning" : {
      "value" : "REGULAR_MAINTENANCE_TIME_FOR_SERVICE",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "serviceTrigger" : {
      "value" : "CALENDAR_TIME",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "engineHoursToService" : {
      "value" : 527,
      "unit" : "h",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "distanceToService" : {
      "value" : 25391,
      "unit" : "km",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "washerFluidLevelWarning" : {
      "value" : "NO_WARNING",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "timeToService" : {
      "value" : 0,
      "unit" : "days",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    }
  }
}
Feb 19 15:27:41 volvo2mqtt [107] - DEBUG: Response status code: 200
Feb 19 15:27:41 volvo2mqtt [107] - DEBUG: {
  "data" : {
    "serviceWarning" : {
      "value" : "REGULAR_MAINTENANCE_TIME_FOR_SERVICE",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "serviceTrigger" : {
      "value" : "CALENDAR_TIME",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "engineHoursToService" : {
      "value" : 527,
      "unit" : "h",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "distanceToService" : {
      "value" : 25391,
      "unit" : "km",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "washerFluidLevelWarning" : {
      "value" : "NO_WARNING",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    },
    "timeToService" : {
      "value" : 0,
      "unit" : "days",
      "timestamp" : "2024-02-19T14:25:08.542Z"
    }
  }
}

What do you mean with "you don't see most of the sensors" ?

Same issue like that one https://github.com/Dielee/volvo2mqtt/issues/171
Only the entities I posted in the log excerpt are seen. All others like mileage, fuel, etc .... are GONE. Just vanished.

I checked which entities are missing / not getting any update from Volvo. These are the ones:
battery_charge_level
charging_connection_status
charging_system_status
electric_range
estimated_charging_finish_time
estimated_charging_time

Hello @iz3man,
The entities, you are missing or that do not receive an upodate, are exactly the endpoints from the Energy API.
See: https://developer.volvocars.com/apis/energy/v1/overview/
After a period that I only received an empty object, when querying any of the Energy endpoints, it started to work.
From February 14, about 15:11 it failed again,but now with a 404 error and a message "NOT_FOUND", with a description "Car model is not supported"
I have a XC40 PHEV model year 2023. What is the model of your car?
Regards

Yep. About a week ago troubles started.
My wife's car is a 2021 XC40 T4 Recharge PHEV.

That's bad. It's the only sensor I'm interested in :) Knowing the state of charge, and when/if to charge and when to stop. :(

Hi,
I also don't get any data besides the config data.
Volvo XC40 Recharge 2024

@Dielee Linus, is this something that the user has to solve? Is only the XC40 Recharge affected by that issue? Anything we can do? Is it up to Volvo to change anything (unlikely) or do you (or someone else) alter the Add-On?
Thanks in advance.

We are completely on V2. So if your car does not work with volvo2mqtt, this can't be fixed by myself. It's a volvo api issue, sorry!

Thanks for the fast reply. But I don't get it ;) The iphone App work, the car works, all other Volvo cars work, and just the XC40 is not sending some data? Is that what you're saying? That would mean that Volvo got seperate APIs for different makes?
If this is the case is there some kind of debugging that can be done from my side? Where would I start.
I don't see ANY chance at all that Volvo would change ANYTHING, as their cars are working, and the apps as well.

The apps are using another API, they are connected via MQTTS, not via REST.
My add-on uses the official API. If your car does not work with their REST API, there is nothing I can do for you.
You can try to intercept the app traffic.. If this works, I don't know.

Ok. Maybe you can help my on working with the API then please.

Where do I get the access token from? And what for do I need it. I got the API token entered, logged in the Volvo online account, but everytime I want to do a query I get this response:

Thank you!

Hello @iz3man,
You asked:

is this something that the user has to solve?

No, it is the responsibility of the developers to switch to version 2, before API version 1 expires (April 30th 2024).
The intention of my post was to issue a warning to Linus, although I had already seen that he was on version 2.

Is only the XC40 Recharge affected by that issue?

No, every Volvo is affected by a change in the API, if they use it.
So it is car independent. However the effect may be different from car to car.

Anything we can do?

No, as an end user you can do nothing. Even we, as developers, have no influence on what Volvo makes available i9n their API's and for which car.

But I don't get it ;) The iphone App work, the car works, all other Volvo cars work.

As Linus says the app uses a different API/endpoint and I believe that your Iphone works. My Android phone works too.
All other cars work? Yes my car works, but I do not receive data from the Energy API, while a month ago I did.
 I sent an email, but they have not responded yet. And I do not expect an answer, because these API's are not intended for end users in order to retrieve their own car data.

just the XC40 is not sending some data?

The XC40 is sending data .
If you use the old "molobrakos" plugin (for HA). See: https://github.com/molobrakos/volvooncall, you will see, that for your own XC40 PHEV you get:
   "hvBattery": {         "hvBatteryChargeStatusDerived": "CableNotPluggedInCar",         "hvBatteryChargeStatusDerivedTimestamp": "2024-03-15T12:59:11+00:00",         "hvBatteryChargeModeStatus": null,         "hvBatteryChargeModeStatusTimestamp": null,         "hvBatteryChargeStatus": "PlugRemoved",         "hvBatteryChargeStatusTimestamp": "2024-03-12T16:33:22+00:00",         "hvBatteryLevel": 94,         "hvBatteryLevelTimestamp": "2024-03-15T12:59:11+00:00",         "distanceToHVBatteryEmpty": 37,         "distanceToHVBatteryEmptyTimestamp": "2024-03-15T12:59:11+00:00",         "hvBatteryChargeWarning": "UnexpectedPowerloss",         "hvBatteryChargeWarningTimestamp": "2024-03-07T12:30:31+00:00",         "timeToHVBatteryFullyCharged": 0,         "timeToHVBatteryFullyChargedTimestamp": "2024-03-12T15:37:53+00:00"     },
Did you try this application?
This is the one, I still use in my production system.

That would mean that Volvo got seperate APIs for different makes?

No, this is not the case.

I don't see ANY chance at all that Volvo would change ANYTHING, as their cars are working, and the apps as well.

Yes, they are working on their API's. Soon your phone app will require 2FA. And I expect more changes.

The apps are using another API, they are connected via MQTTS, not via REST.

@Dielee . I did not do any research so far, but if the app uses MQTTS, then there is a broker.
Would that be something to look at. (Try to find the IP address?)

You can try to intercept the app traffic.. If this works, I don't know.

I was successful with that with the old Volvo on Call app (See: https://github.com/molobrakos/volvooncall/issues/52)
With the latest Volvo Cars app it returned an error and I was not able to capture the data packets.

Where do I get the access token from? And what for do I need it. I got the API token entered, logged in the Volvo online account, but every time I want to do a query I get this response:

As said earlier the Volvo API's are not intended to be used by hobbyists, like you and me. Probably that's why, we do not get an answer on mails and requests for publication. I wish I could a valid key, etc in a legal way/ Volvo want as partner a "legal entity" and unfortunately private persons are not considered as such.  So all "open source developers" has to find another route.
You can do as well, but remember the Volvo servers are located in Sweden and in Sweden they have already since July 1st. 1973, a law against data trespassing. So breaking into their servers is illegal and may cause trouble.
study their documentation carefully,
Regards

You said, that you are completely on V2.
But my guess is, that you are just requesting the values of the Connected Vehicle API
But the values, that are missing are from the Energy API.
https://developer.volvocars.com/apis/energy/v1/specification/

Hi,
I think, there's a deviation in the definition of the V2 specification.
i the specification, the charge level is sopposed to be sent via
https://api.volvocars.com/connected-vehicle/v2/vehicles/{vin}/fuel
But if you go to the V2 sandbox, the topic is
https://api.volvocars.com/connected-vehicle/v2/vehicles/{vin}/battery-charge-level

Hello @schrej-zz,
First regarding API versions
For normal users (Non Volvo employees) 4 different API's are available.

Extended API > Still on version v1
Connected API > Version 1 will be deprecated April 30, 2024 and is replaced by version 2
Energy API > Still version 1
Location API > Still version 1

I think we might expect an update of the Location API, if they want to bring the API header in line with the header of the other API's. This was also one of the differences between v1 and v2 of the Connected API.
You said:

I think, there's a deviation in the definition of the V2 specification.
i the specification, the charge level is supposed to be sent via
https://api.volvocars.com/connected-vehicle/v2/vehicles/{vin}/fuel

This is one of the things that recently changed.
The fuel amount, you could retrieve in the Connected API V1 with the scope: openid and  conve:fuel_status and an endpoint fuel, while the Battery Charge Level had the same scope, but as endpoint: battery-charge-level.
But it moved to version v2. In the beginning they did not change the scope, so that you could retrieve both values from the same endpoint: "fuel".
Then suddenly Volvo introduced a new scope for the fuel: openid, conve:fuel_status and conve:battery_charge_level.
I mentioned that in an earlier post.
So instead of 1 scope and 2 endpoints, they moved to 2 scopes and 1 endpoint in v2.
For more info see: https://developer.volvocars.com/apis/connected-vehicle/v2/release-notes/
So the sandbox is using the old v1 endpoint.
And now our problem:  The latest added scope (conve:battery_charge_level) is not in the list of authorized scopes.
Until Volvo is more "open" and "communicative", there is not much developers can do.
Regards

Thanks @FireWizard52 . I guess I understand it a little better now, and see there's nothing I can do. Unfortunately integrating a PHEV in HomeAssistant w/o knowing the SOC is not possible. So I hope time will bring the solution :)
Thanks everyone for contributing - very much appreciated.
And offtopic: 2FA for the Volvo app? I hope not - or at least not mandatory. 2FA is a plague of it's own.

@iz3man

And offtopic: 2FA for the Volvo app? I hope not - or at least not mandatory. 2FA is a plague of it's own.

Yes, I got a message in the app, February 22, announcing that.  I expect it is mandatory, but hope you can stay logged in, like now.

This morning, Monday, March 18, 2024, since 11:00h the "Old"VoC server is not accessible anymore.
It returns:
$ voc status
Could not connect to the server.
I'm sure that 2FA is the reason. I logged out of my Volvo Cars app and when I tried to log-in again, I had to submit a
Volvo ID-verificationcode, consisting of 6 digits. Currently it is impossible to make a connection with the Volvo ID-service.
So this is probably the end of the "old" Volvo integration, unless someone is able to solve this issue.
Thanks to all contributors.
 Why not ask the customer what the want/need. What are API keys for then? How should ANY automation work with mandatory 2FA?? Using 2FA for starting the heating? No way.

Hello @iz3man,
Do not react so angry. It is quite common today that for security reasons 2FA is used. What would be the security for your bank account or your credit-card account without 2 FA?
Up to now (today) I never logged in with my app. I did it once (or maybe a few times after) and when the app connected, I stayed logged-in. I tested it now, by logging-out and the logging-in again.  And now I have to add the Volvo-ID-verification code, which has been sent by email. This means that the "old" (molobrakos) version stopped working.
I expect that as soon the Volvo  ID-service is up and running you simply log-in again and that it stays logged-in.
It will have consequences for applications that use the same API or an API with the same functionality.
But here @Dielee is working on a solution (See #180) for that and other applications will soon follow.
So in the Volvo app it will not have serious consequences, except that you have to submit a code during log-in, but that will last until you log-out again. Other applications will come with a solution.
Regards

May your words be true :) You know I'm 50+ now, and would call myself a digital native, but for my parents many things are unuseable in the meantime. I get calls from my parents all the time ... ;)

I hope so, You are 50+, well let me tell you I'am 70+, but working since the seventies of the last century with computers, etc :)
I can image that those older people has problems. It's the same with me. I do not understand my grandson and his games either :)


fix: volume data integrity check

What problem are we solving?
It wants to check the last 10 chunks, but actually only checks the last chunk and then exits.
How are we solving the problem?
How is the PR tested?
Checks

[ ] I have added unit tests if possible.
[ ] I will add related wiki document changes and link to this PR after merging.

This caused tests failing https://github.com/seaweedfs/seaweedfs/actions/workflows/go.yml
=== RUN   TestLDBIndexCompaction
I1017 11:00:14.025686 volume_loading.go:137 checking volume data integrity for volume 1
I1017 11:00:14.026303 volume_loading.go:170 loading leveldb index /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.ldb
I1017 11:00:14.046553 needle_map_leveldb.go:122 generateLevelDbFile /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.ldb, watermark 0, num of entries:0
I1017 11:00:14.065913 needle_map_leveldb.go:66 Loading /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.ldb... , watermark: 0
I1017 11:00:14.243552 needle_map_leveldb.go:338 loading idx to leveldb from offset 0 for file: /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.cpx
    volume_vacuum_test.go:92: compaction speed: 55122659.95 bytes/s
I1017 11:00:14.538781 volume_vacuum.go:114 Committing volume 1 vacuuming...
I1017 11:00:14.638140 needle_map_leveldb.go:338 loading idx to leveldb from offset 9697 for file: /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.cpx
I1017 11:00:14.734931 volume_loading.go:96 readSuperBlock volume 1 version 3
I1017 11:00:14.734983 volume_loading.go:137 checking volume data integrity for volume 1
W1017 11:00:14.734993 volume_checking.go:118 data file /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.dat actual 15968744 bytes expected 15968712 bytes!
I1017 11:00:14.735850 volume_loading.go:140 volumeDataIntegrityChecking failed data file /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.dat actual 15968744 bytes expected 15968712 bytes
I1017 11:00:14.735862 needle_map_sorted_file.go:24 Start to Generate /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.sdx from /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.idx
I1017 11:00:14.778047 needle_map_sorted_file.go:26 Finished Generating /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.sdx from /var/folders/ly/_jjpz8l54jg632bw5wct_40m0000gp/T/TestLDBIndexCompaction3984692687/001/1.idx
--- FAIL: TestLDBIndexCompaction (0.76s)
panic: interface conversion: interface {} is *storage.SortedFileNeedleMap, not *storage.LevelDbNeedleMap [recovered]
	panic: interface conversion: interface {} is *storage.SortedFileNeedleMap, not *storage.LevelDbNeedleMap

Reverting this. The code is meant to find the first valid entry reversely.

the function name may be misleading.


"Protocol error" when attempting to connect from Microsoft's official Android RDP client

I tried to connect to an XRDP server from my Android phone, using Microsoft's Remote Desktop app for Android. Both times I tried, it failed to connect, telling me a protocol error occurred. Considering this is an official client by the same company that created the "standard", and not just some random program someone created, I think it's important that XRDP is compatible.

Duplicate of #246?

Oh yep, that's the error I'm getting. I didn't see that when I looked through the issues. Thanks.


Fix top menu - screen shot attached

After adding current and future plansto menu, text starts stacking when you resize the window

We need to fix but have you tried making the window bigger? That should be a temporal solution ...

I will try next time it happens
On Mon, Mar 21, 2016 at 8:30 AM, Antonio Gonzalez<EMAIL_ADDRESS>wrote:

We need to fix but have you tried making the window bigger? That should be
a temporal solution ...
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/biocore/qiita/issues/1699#issuecomment-199340806

--
Gail Ackermann
Knight Lab
UCSD
<EMAIL_ADDRESS><EMAIL_ADDRESS>


After starting the Aspire configuration, nothing happens

This bug seems to be introduced in the latest release. As a quick fix, you can remove a Build Project BeforeLaunchTask and add a Build Solution one.

I can confirm that the workaround works, thank you!


Lorebook access

Should there be a way to access the lorebook besides a keyboard shortcut that isn't set by default? Am I missing a way of adding it to a sidebar, or something?

There's no other way right now. You can only access it through commands/shortcuts.


update endpoint status

Implements #325

This change is

/hold
@yuzisun  ready for review, though I've held it as I need to add some more tests

/hold cancel
tests look good to me :)


[IOS] Cannot sign in again after first revokeAccess and logout

Hello,
For these 2 weeks on IOS app always got random condition using GoogleSignIn, sometimes normal using google sign in but sometimes after success login (got serverAuthCode) for the first time after that cannot sign in again with serverAuthCode response was null
Steps to Reproduce

Login normally using react native google sign in  and got serverAuthCode for the first time
After success login revokeAcess and logout
Login again and always got response null from serverAuthCode

Expected Behavior
Could sign in normally after a few times using google sign in and always got response from serverAuthCode
Actual Behavior
Got response null from serverAuthCode and cannot sign in again using Google sign in
Environment

react-native version 0.60.4
react-native-google-signin version 2.0.0
Pods :
RNGoogleSignin (2.0.0)
GoogleSignIn (4.4.0)
GTMSessionFetcher/Core (1.1)
IOS 13

hello and thanks for reporting. I recommend you upgrade to latest version of the package first to find out if the issue is still there. Can you please do that and then get back to me? Thanks

success after upgrade into latest version, Thanks


AttributeError: module 'modal' has no attribute 'forward'

while running https://modal.com/docs/examples/serve_streamlit#sessions

Can you try importing forward like explained here?

same behaviour, was it was depreciated in the latest modal-client version?


Terraform 0.13 compatibility

See Upgrading to Project Factory v10.0
Need to fix this:
Error: Provider configuration not present

To work with
module.host.module.project.module.project-factory.null_resource.preconditions
its original provider configuration at
provider["registry.terraform.io/-/null"] is required, but it has been removed.
This occurs when a provider configuration is removed while objects created by
that provider still exist in the state. Re-add the provider configuration to
destroy
module.host.module.project.module.project-factory.null_resource.preconditions,
after which you can remove the provider configuration again.

Steps to reproduce:

Terraform v0.13.6
terragrunt version v0.27.3

cd ~/workspace/v6/platform/terraform/multinic/10_project
git checkout jm/38_tf13
cowboy terragrunt init -upgrade
cowboy terragrunt plan

Upgrade process:

For the project creation, apply project factory 9.2 and network 2.0 using TF 0.12.
Apply the same with TF 0.13
Update to project factor 10.x and network 3.x
Apply the same with TF 0.13

Note this process avoids the above Provider configuration not present error.

Released in 4.0.0.  Version 3.5.0 is the last version to support Terraform 0.12 and should be applied cleanly prior to updating to version 4.0.0 of this module.


Update README.md

This doesn't work so I'm removing the reference to it.

I am fairly certain that I tested it in several setups for all architectures in this PR https://github.com/CUNY-CL/yoyodyne/pull/74 and it worked? Either my memory is wrong or maybe we have a regression?

I actually didn't test it---we jsut have a TODO that says it's not working:
https://github.com/kylebgorman/yoyodyne/blob/61fbb9a085d081c64736343cdb51225765f736cc/yoyodyne/predict.py#L53
If it is working then we should revert this and remove that TODO.

Ah, I should have removed that TODO then! I can test to be sure.

Closing in favor of #97.


Update PR description to add feature checks and possible screenshots/videos

Please verify the following:

[x] yarn test jest tests pass with new tests, if relevant
[x] yarn lint eslint checks pass with new code, if relevant
[x] yarn format:check prettier checks pass with new code, if relevant
[x] README.md (or relevant documentation) has been updated with your changes

Describe your PR
This PR updates Ignite's PR template with:

Add a new check to enforce maintainers to test features
Add new section for before/after screenshots

:tada: This PR is included in version 10.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Preserve ICC metadata from input images

Hi,
Thank you for making this tool! It's very handy and I look forward to its future development. I ran into an issue, though:
For palettes saved in images with an embedded color profile, the analysis seems fine but the output image looks off because plot.png does not share the same color profile as the input. [Edit: Or the analysis is off and the output plot profile should be sRGB. I don't know, it's been a long day.]

Just in case there's any issue with the display of the images above, the input png was built from this GPL file, then had a profile assigned to it:
GIMP Palette
Name: RYB
Columns: 4
255 0 0 FF0000 1
255 106 0 FF6A00 2
255 162 0 FFA200 3
255 207 0 FFCF00 4
255 255 0 FFFF00 5
129 212 26 81D41A 6
0 169 51 00A933 7
21<PHONE_NUMBER>66 8
17 89 166 1159A6 9
60 42 146 3C2A92 10
105 12 133 690C85 11
170 0 85 AA0055 12

The yellow green, for example, is #81D41A, but looks like #9dd339.
Thank you.

Hello! At the moment, the analyser doesn't read or write ICC metadata when handling images and relies only on RGB values provided by the image library.
Currently, the library seems to be unable to handle ICC profiles: https://github.com/image-rs/image/issues/1030
I guess using something like img-parts in addition to image just for reading/writing ICC metadata is the optimal way for now. I'll try that.

By the way, if you also want ICC metadata to affect how RGB values are converted to XYZ and forth, open a separate issue - it'll require actual interpretation of the profile.

The issue should be resolved now; imagemagick's identify -verbose shows that the ICC profile of the analysis is the same as the one of your example. Though it's preserved only for PNG and JPEG at the moment (restrictions of img-parts). Reopen the issue if you encounter additional problems with the feature.


Converted tf.gfile.GFile to tf.io.gfile.GFile

System information

OS Platform and Distribution (Google Colab):
TensorFlow version (tensorflow 2.6.0):

I was working with the tensorflow's object detection api and as soon as i ran the following code:
PATH_TO_LABELS = '/content/models/research/object_detection/data/mscoco_label_map.pbtxt'
category_index = label_map_util.create_category_index_from_labelmap(PATH_TO_LABELS, use_display_name=True)
It showed the following error.
AttributeError                            Traceback (most recent call last)
 in ()
1 PATH_TO_LABELS = '/content/models/research/object_detection/data/mscoco_label_map.pbtxt'
----> 2 category_index = label_map_util.create_category_index_from_labelmap(PATH_TO_LABELS, use_display_name=True)
2 frames
/usr/local/lib/python3.7/dist-packages/object_detection/utils/label_map_util.py in load_labelmap(path)
130   Returns:
131     a StringIntLabelMapProto
--> 132   """
133   with tf.io.gfile.GFile(path, 'r') as fid:
134     label_map_string = fid.read()
AttributeError: module 'tensorflow' has no attribute 'gfile'
I tried to change the load_labelmap file too but it showed no result. Viewing this on stack overflow suggested that I should downgrade my tensorflow version to 1.x but for the particular program, I need to have the tensorflow version above 2.5
What can be the possible solution to this problem.

@Bhavay192 ,
Could you please have a look at this  SO link 1 and comment from the issue 2 with the similar error.It helps.Thanks!

Downgrading to 1.x is a bad advice.
Please make sure your TF version is 2.x and the dependencies you are using are also using TF 2.x.
Based on
/usr/local/lib/python3.7/dist-packages/object_detection/utils/label_map_util.py in load_labelmap(path)
130 Returns:
131 a StringIntLabelMapProto
--> 132 """
133 with tf.io.gfile.GFile(path, 'r') as fid:
134 label_map_string = fid.read()

AttributeError: module 'tensorflow' has no attribute 'gfile'

(from your error log, but formatted properly), it seems the error comes from object_detection wheel.

Hey @tilakrayal , I will look into the links and let you know about it. Thanks

https://pypi.org/project/object-detection-0.1/#files is not a TF product. So we should not handle this in this repository.
Looking at the package, it lacks a setup.py, so it does not specify what version of TF it needs. The code in the file assumes TF 1.x
  with tf.gfile.GFile(path, 'r') as fid:
    label_map_string = fid.read()
    label_map = string_int_label_map_pb2.StringIntLabelMap()
    ...

I would recommend not using this package as it only works with TF 1.x it seems but TF 1.x is no longer supported.
Closing this, since this is not a TF issue.


Support for other (local) stores rather than Google Play

Hi
I'm doing a project with RN that needs in app billings and we have to publish this app on a store rather than GP and I have to use it's IAB API (The reason is that I want to support our country currency)
I asked around and it seems like their API behaves the same way as Google Play

and this is the complete docs in English: http://developers.cafebazaar.ir/en/docs/iab/
My question is about how/where should I change this repository to use Bazar's API instead of Google Play?
Thanks,
Alireza

Hi @Qti3e
This library wraps https://github.com/anjlab/android-inapp-billing-v3, so the API surface of it matches the wrapped library. For that reason, I do not see any easy way to change which store is used.

Ok,
Thanks for your answer, I'll try to do something with that.


How to disable "Motion event recordings"?

Isn't it possible to fully disable the recording of motion events? Is the configuration done within the web interface not fully implemented yet? The camera works well and I could connect two of the with my motioneye on Raspberry Pi. But I dont want to record motion events on the SD card of the cameras.
Thanks in advance and also for the great work on that hack!
Best
Marco

The hack doesn't allow to switch off the recording.
But probably you can do it with the app, I don't remember exactly.
If using the app is not possible, you could try to kill mp4record.
Create a shell script
/tmp/sd/yi-hack/startup.sh
with this content:
#!/bin/sh

killall mp4record

mp4record will be restarted within a few seconds by watch_process unless you kill that too.
mp4record can be blocked by creating a file:
mv /tmp/sd/record /tmp/sd/record.bak
touch /tmp/sd/record

mp4record will be restarted within a few seconds by watch_process unless you kill that too.
mp4record can be blocked by creating a file:
mv /tmp/sd/record /tmp/sd/record.bak
touch /tmp/sd/record

mp4record will be restarted within a few seconds by watch_process unless you kill that too.
mp4record can be blocked by creating a file:
mv /tmp/sd/record /tmp/sd/record.bak touch /tmp/sd/record

Does that actually disable reads/writes to the sd card, or just keep it looping and reading/writing to the card and failing, instead?
My instinct is that it just fails over and over again, but it doesn't appear to be using the sd card.

I don't think so. mp4record obviously checks whether /tmp/sd/record exists and tries to write a file in it, which immediately fails in open() because record itself is a file, or it realizes that record ist not a directory and tries to mkdir() it which immediately fails as well. In both cases the SD card is not written to.

Related to this "feature request", there is some news with all changes comes in the last release? Can be disabled motion recording without disable the camera at all?
I use this hack with home-assistant and I need, for a "privacy compromise" to disable recording, but I want that live video continue to work for using it ad "baby monitor", when baby sleep or stay alone in some parts of house.
Thank you.

Related to this "feature request", there is some news with all changes comes in the last release? Can be disabled motion recording without disable the camera at all?

Try this.
Create a shell script
/tmp/sd/yi-hack/startup.sh
with this content:
#!/bin/sh
killall watch_process
killall mp4record

I use this hack with home-assistant and I need, for a "privacy compromise" to disable recording, but I want that live video continue to work for using it ad "baby monitor", when baby sleep or stay alone in some parts of house.

If you use the "Switch on/off the camera" switch, this should disable local recording. I don't remember exactly but it should.


how ignore some part of code when run test.

I'm using materialize and I need to add the scripts in the index file.

I'm using create-react-app, so the problem is when I run the test $ is not defined

I add the jquery $ as a global var and now the problem is .sideNav is not defined

how can say jest ignore the function in the image 2 or some part of code?

Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions. Thank you :)

You can make use of Jest mocking system:

docs on jest.mock
and here example in Jest repository that you should find useful in this case

However if you experience any troubles, reach out to SO or Discord community.


fix(docker/settings): fetch correct value for allow sysctl

fix EE-633

Close https://github.com/portainer/portainer/issues/5175


reduce JS minified distributable

that's around 50k, way too much. blame goes to faye-browser and buffer (80k and 45k before minification)

current version is 78KBytes :open_mouth:

for the record, Faye is responsible for most of the package size


added Regional Community College Mondriaan of The Hague

@Ronnnn I am very sorry for a delay with processing your request. It is now merged. Thank you!


SURVIVOR merge takes cross chromosomes to merge

Hi
I am trying to merge vcf from breakdancer, delly and lumpy but it seems SURVIVOR taking it wrong while merging. In breakdancer, the call is DEL in chr3 whereas lumpy has chr6 with DUP with more or less same position and SURVIVOR merges this entry as DUP. I think, its bug ??
Can you also explain? how strand information is used while merging the SVtype from VCF?
`## command
ls *.vcf > samples.txt
brekdancer.vcf
delly.vcf
lumpy.vcf
SURVIVOR_v1.0.5
SURVIVOR merge samples.txt 1000 2 1 1 0 50 mergedSV.vcf
Original entry
lumpy
grep '80455101' lumpySVOut/*sort.vcf
chr6	80455101	40289	N		.	.	SVTYPE=DUP;STRANDS=-+:9;SVLEN=6442;END=80461543;CIPOS=0,0;CIEND=0,0;CIPOS95=-1219,2;CIEND95=-2,1327;IMPRECISE;SU=9;PE=9;SR=0	GT:SU:PE:SR	./.:9:9:0
Breakdancer
chr3	80455996	.	.	.	.	PASS	PROGRAM=breakdancer;SVTYPE=DEL;SVLEN=5547;SVEND=80461543	GT:DP	1/.:0
Results
Problemetic merged Entry:
it comes from BreakDancer and Lumpy
chr3	80455101	DUP004590SUR	N		.	PASS	SUPP=2;SUPP_VEC=101;AVGLEN=-5994.5;SVTYPE=DUP;SVMETHOD=SURVIVORv2;CHR2=chr3;END=80461543;CIPOS=0,895;CIEND=0,0;STRANDS=-+	GT:PSV:LN:DR:ST:TY:CO	1/.:NA:5547:0,0:+-:DEL:chr3_80455996-chr3_80461543	./.:NaN:0:0,0:--:NaN:NaN	./.:NA:6442:0,9:-+:DUP:chr3_80455101-chr3_80461543'

Hi again,
Thank you so ,much for your gentle reply. I will work on this, f its work then fine otherwise  will take just start position from all different tool and compare it since the start, end gives the position of the insertion or the region were the insertion occurs. I feel really don't need SURVIVOR to do this job.
By the can you guide me, how to calculate Allele Frequency along with homo and heterozygous from Merge vcf ? I was thinking to take DR (supporting reference,variant reads in that order) and calculate the Allele freq.
all freq = alt read/ref reads+alt reads.
Is that fine? Since, delly have only heterozygous and homozygous information, so i guess i have to stick with it.
##output from survivor
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
##FORMAT=<ID=PSV,Number=1,Type=String,Description="Previous support vector">
##FORMAT=<ID=LN,Number=1,Type=Integer,Description="predicted length">
##FORMAT=<ID=DR,Number=2,Type=Integer,Description="# supporting reference,variant reads in that order">
##FORMAT=<ID=ST,Number=1,Type=String,Description="Strand of SVs">
##FORMAT=<ID=QV,Number=1,Type=String,Description="Quality values: if not defined a . otherwise the reported value.">
##FORMAT=<ID=TY,Number=1,Type=String,Description="Types">
##FORMAT=<ID=ID,Number=1,Type=String,Description="Variant ID from input.">
##FORMAT=<ID=RAL,Number=1,Type=String,Description="Reference allele sequence reported from input.">
##FORMAT=<ID=AAL,Number=1,Type=String,Description="Alternative allele sequence reported from input.">
##FORMAT=<ID=CO,Number=1,Type=String,Description="Coordinates">

#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO	FORMAT	default	delly	lumpy
chr17	39848828	130992	N	<DEL>	.	PASS	SUPP=2;SUPP_VEC=011;SVLEN=-31114809;SVTYPE=DEL;SVMETHOD=SURVIVOR1.0.6;CHR2=chr17;END=70963595;CIPOS=0,2;CIEND=0,85;STRANDS=+-
GT:PSV:LN:**DR**:ST:QV:TY:ID:RAL:AAL:CO	./.:NaN:0:0,0:--:NaN:NaN:NaN:NAN:NAN:NAN	 0/1:NA:31114850:97,0:+-:.:DEL:DEL00093894:NA:NA:chr17_39848830-chr17_70963680	./.:NA:31114767:0,30:+-:.:DEL:130992:NA:NA:chr17_39848828-chr17_70963595

Just to be clear SURVIVOR also compares the size of the insertion and does the whole thing for every type of variant. If you are just focused on locations of ins you are right.
So allele count (across samples) can be taken from SUPP=. However, this does not take the 0/1 or 1/1 into account. If you want to take this into account you need to write your own script. The allele frequency within the sample is exactly what you suggest.
Cheers
Fritz

Thank you very much Fritz for your help and support.


[SPARK-SQL][SPARK-32977] Fix JavaDoc on Default Save Mode

The default is always ErrorsOnExist regardless of DataSource version. Fixing the JavaDoc to reflect this.

Can one of the admins verify this patch?

@brkyvz  - Could you please review this re: our conversation a while back :)

@rdblue  You may want to take a look as well if you have a moment

+1

Test build #129037 has started for PR 29853 at commit 9bcb494.

Some random image reading thing failed in Pyspark tests, can't imagine that it's related ;)

Test build #129037 has finished for PR 29853 at commit 9bcb494.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129037/
Test PASSed.

Yeah, that's a known flaky test.

It would have been better if we keep the PR template though.


All Video files fail to find Subtitles

A few of the Subtitle websites are blocked in my country, so I'm aware that these sort of Githubs only work on a VPN, but,
I'm still encountering problems such as ;
"The selected file cannot be used to find subtitles"
https://i.imgur.com/68eJ0Rl.png
Does anybody have some input as to why it can't find Subtitles for every single Movie/Video I'm inputted?

The error you're encountering might be because the SubDB API is currently down and inactive. Since the service is offline, it’s unable to fetch any subtitles, which causes issues even if you're using a VPN. Unfortunately, without the API being available, this tool won’t be able to retrieve subtitles for any videos.


Add gcc-arm-embedded repository

This is needed to cross-compile for arm-none-eabi.

Duplicate of #132 and #219. We really need this for my students.


add component governance file

This is the new way of registering components.
More info here: https://docs.opensource.microsoft.com/tools/cg/features/cgmanifest/

[ ] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
[ ] Title summarizes what is changing.
[ ] Has a news entry file (remember to thank yourself!).
[ ] Appropriate comments and documentation strings in the code.
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[ ] Unit tests & system/integration tests are added/updated.
[ ] Test plan is updated as appropriate.
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed).

Interesting, so this replaces the process on the website that wasn't working anymore? I see this cgmanifest in the python extension already.

@IanMatthewHuff yes

The guidance for the file you added was for dependencies that can't be picked up elsewhere:
A cgmanifest file is a json file used to register components manually when the component type is not supported by governance.

@rchiodo they asked me to do it like this: https://mseng.visualstudio.com/AzureDevOps/_workitems/edit/1849128


AssertionError: Engine world size (2) != Runtime world size (1)

    assert world_size == tensorrt_llm.mpi_world_size(), \         f'Engine world size ({world_size}) != Runtime world size ({tensorrt_llm.mpi_world_size()})'
How to modify tensorrt_llm.mpi_world_size when I generate tp=2 engine and testing with "python llama/summarize.py" , mpirun -n 2 is not the choice neither.

Can you share the scripts to reproduce your issues?

Can you share the scripts to reproduce your issues?

By the way, how to specify the maximum output length when starting triton server because there is always a warning: "Number of requested output tokens (1024) exceeds maximum output length (512). Number of requested output tokens is changed to 512"

Can you open another issue to prevent confusion.


treewide: GitHub Action failures

Recently, some (often only the last one) of the testbed GitHub Actions fail, and I manually re-run them to success. Ideally, this should be resolved.
Usually they fail due to timeout or somehow losing internet connection or not being able to download something.
I know GitHub Actions have an expected low probability failure rate. But now it happens nearly all the time.
Here is one example: https://github.com/danth/stylix/actions/runs/9268432783/attempts/1.

Could this be related to the Magic Nix Cache action?

Closing this issue, as it seems like the GitHub Actions are no longer timing out.


Not manually configured navigation

Not going to have looking for pages automatically so I can't modify it


type: [String] throws invalid definition error

Hi,
Im having an issue creating my schemas with the node version of the package, the following throws an invalid definition field may not be an array error.
videos: {
    type: [String],
    label: 'Videos',
    defaultValue: []
},

Is this not supported anymore on this version of simple schema?
Full error
W20170219-12:31:24.073(-5)? (STDERR) Error: Invalid definition for videos field: "type" may not be an array. Change it to Array.
W20170219-12:31:24.073(-5)? (STDERR)     at /Users/wanchope/Proyectos/CoStartup/trovaMeteor/node_modules/simpl-schema/dist/SimpleSchema.js:850:13
W20170219-12:31:24.073(-5)? (STDERR)     at Array.forEach (native)
W20170219-12:31:24.073(-5)? (STDERR)     at checkAndScrubDefinition (/Users/wanchope/Proyectos/CoStartup/trovaMeteor/node_modules/simpl-schema/dist/SimpleSchema.js:843:39)
W20170219-12:31:24.074(-5)? (STDERR)     at /Users/wanchope/Proyectos/CoStartup/trovaMeteor/node_modules/simpl-schema/dist/SimpleSchema.js:415:50
W20170219-12:31:24.074(-5)? (STDERR)     at Function._.each._.forEach (/Users/wanchope/Proyectos/CoStartup/trovaMeteor/node_modules/underscore/underscore.js:158:9)
W20170219-12:31:24.074(-5)? (STDERR)     at SimpleSchema.extend (/Users/wanchope/Proyectos/CoStartup/trovaMeteor/node_modules/simpl-schema/dist/SimpleSchema.js:414:28)
W20170219-12:31:24.075(-5)? (STDERR)     at new SimpleSchema (/Users/wanchope/Proyectos/CoStartup/trovaMeteor/node_modules/simpl-schema/dist/SimpleSchema.js:94:10)
W20170219-12:31:24.075(-5)? (STDERR)     at meteorInstall.imports.api.Startups.index.js (imports/api/Startups/index.js:18:19)
W20170219-12:31:24.075(-5)? (STDERR)     at fileEvaluate (packages/modules-runtime.js:188:9)
W20170219-12:31:24.075(-5)? (STDERR)     at Module.require (packages/modules-runtime.js:111:16)

I think you can use [String] only in shorthand definitions.
Shorthand
{
  friends: [String],
}

Longhand
{
  friends: { type: Array },
  'friends.$': { type: String },
}

I took this example from the docs: https://github.com/aldeed/node-simple-schema#more-shorthand

You are correct @ovidyou, I was wondering if this was intentional or not, because in the Meteor version it is possible to use [String] in both ways.

@indesignlatam Yes, an intentional change. When you specify additional options besides type, the package then had to guess whether you meant them to be on the array or the array items. It also caused a lot of confusion. So now it's supported only in the shorthand where guessing isn't necessary. (Obviously still causing some confusion, but hopefully less.)


clean up notebook regex

For example, we should remove all instance of n_epochs_all and also remove the first cell stuff.
We should also be able to remove all code that has conditionals if in test mode.

Not relevant as of scvi-tools release.


conflict with https://github.com/ajoberstar/reckon gradle plugin

I've added restdocs-openapi plugin to my build.gradle and now building fails with:
> Failed to apply plugin [id 'com.epages.restdocs-openapi']
   > Could not create an instance of type com.epages.restdocs.openapi.gradle.RestdocsOpenApiPluginExtension_Decorated.
      > org.ajoberstar.reckon.gradle.ReckonPlugin$DelayedVersion cannot be cast to java.lang.String

Do you have a sample project so I could reproduce the case to have a deeper look?

here it is: https://github.com/thugcee/gradle-recon-restdocs-openapi-test

Can you try release 0.2.3 - there we are more careful when accessing the project version property.

Yes, it works now. Thank you!


Exceptions thrown by Command handlers are not handled properly

When exceptions are thrown in execution handlers, they are currently not handled properly. The asynchronous handler doesn't catch exceptions at all, in fact:
https://github.com/Incendo/cloud/blob/1.2.0-dev/cloud-core/src/main/java/cloud/commandframework/execution/AsynchronousCommandExecutionCoordinator.java#L95
The annotation handler merely prints the stack trace. No message is sent to the player informing him an internal error occurred:
https://github.com/Incendo/cloud/blob/1.2.0-dev/cloud-annotations/src/main/java/cloud/commandframework/annotations/MethodCommandExecutionHandler.java#L119
At the very least some sort of message should be displayed to the player when command execution fails. Ideally, the exception propagates to the Command Execution Coordinator, where it could possibly be handled by custom implementations.
The exception could be wrapped into a special 'Command Execution Exception'.
I made some changes in a personal fork, but based on discussion in Discord it is probably not sufficient.
https://github.com/Incendo/cloud/compare/1.2.0-dev...bergerhealer:1.2.0-dev

With the PR merged, this issue can now be closed and I can confirm this works excellently. While we had some disagreement about catching Errors, the main mechanic works fine, which is what matters :+1:
By handling the Command Execution Exception it is possible to handle custom exceptions thrown in command handlers, which is a great way to handle various error conditions and show appropriate messages. This works really well and no unwanted text is logged to console.
Very happy with these changes


Regression tests in CI

The um2nc repository contains a script for running manual regression tests. These tests run the um2nc conversion on a subset of an ESM1.5 output file, and compare the results with the output of the original um2netcdf4.py script. They require you to have a local copy of the ESM1.5 output fields file, and output netCDF from the original um2netcdf4.py script with different options applied.
These files are:
8.2M	aiihca.paa1jan.subset (the fields file)
1.2M	aiihca.paa1jan.subset.nomask.orig.nc (um2netcdf4.py output with --nomask option)
128K	aiihca.paa1jan.subset.orig.nc (um2netcdf4.py output without nomask flag)

For both netCDF files, the --nohist flag was also applied.
I need to update the tests to allow for different versions of hdf5, however they seem like a good candidate for implementing as part of the CI - e.g. I think it would be really valuable if we could run them automatically for each PR into main.
Speaking with @aidanheerdegen, this sounds doable! The main decisions to make are where the fields files + netCDF files should be stored, and where the regression tests should be run. Options mentioned by @aidanheerdegen include storing the data and running on gadi, as with the model config reproducibility tests, or running on github directly with the data stored elsewhere.
Another decision is around the fields file to use – the aiihca.paa1jan.subset file is small, but it only contains two variables (pr and ta_plev), and so some important parts of the code aren't run. The heaviside variables are missing, and as a result the masking step  skipped. This part of the code is not covered by the unit tests, as it involves hard to test iris methods.
If it's viable, I'd be in favour of using a slightly bigger file which requires more of the code to run. The whole aiihca.paa1jan file that aiihca.paa1jan.subset is derived from is 511M, perhaps too big. I'm wondering whether there'd be a possible middle ground, and whether the choice of file determines where the CI tests should run?
For some extra information, the following is the /usr/bin/time -l output from running the binary_diff.sh script on my macbook:
       10.00 real         2.00 user         1.10 sys
           157335552  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
               35988  page reclaims
                 887  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                   0  messages sent
                   0  messages received
                   0  signals received
                1633  voluntary context switches
                4233  involuntary context switches
            18911038  instructions retired
            20490635  cycles elapsed
             2098112  peak memory footprint

The following shows the resource usage when swapping the small aiihca.paa1jan.subset file for the bigger aiihca.paa1jan file.
 16.96 real        15.02 user         1.16 sys
<PHONE_NUMBER>  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
              160969  page reclaims
                2497  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                   0  messages sent
                   0  messages received
                   0  signals received
                5265  voluntary context switches
              109062  involuntary context switches
            19046728  instructions retired
            21810650  cycles elapsed
             2032576  peak memory footprint

@aidanheerdegen, @CodeGat, @atteggiani – your thoughts on the best way to set up the CI tests would be valuable!

Hi @blimlim,
Thank you for raising this issue.
Yes what you talk about is definitely doable. I had a chat with @CodeGat earlier today to better understand what was already implemented for the model config reproducibility tests, and how much of that code can be applied to our case here.
Where to run the tests

The main decisions to make are where the fields files + netCDF files should be stored, and where the regression tests should be run. Options mentioned by @aidanheerdegen include storing the data and running on gadi, as with the model config reproducibility tests, or running on github directly with the data stored elsewhere.

I think the better option is to run the tests (and also store the data) on Gadi (either through a PBS job or on the login node, depending on the "workload" of the tests).
The reasons are multiple:

Main reason would be to set-up a "regression-testing" infrastructure that could be applicable to other repos (for example, we have to do a similar thing with the replace_landsurface and era5_grib_parallel repos)
The input/output files size for testing could get pretty big, making it slower (or impossible) to copy the files somewhere else (for example to a GitHub runner).
If tests require greater computational resources (n. CPUs, MEM, etc.) than the ones available on the GitHub runners, we can have access to them on Gadi through PBS jobs.

I think a good option for storing data is within the vk83 project, maybe in a folder called regression-tests or similar and subdivided for each different project (based on the repo name). In the case of this repo, the whole path could be /g/data/vk83/regression-tests/um2nc-standalone/.
I don't think the number or size of files would be too big in general, making the inodes and size quota not an issue. In any case, we could think about ways to optimize that if problems arise.
Which fieldsfile to use (which tests to run)

Another decision is around the fields file to use – the aiihca.paa1jan.subset file is small, but it only contains two variables (pr and ta_plev), and so some important parts of the code aren't run. The heaviside variables are missing, and as a result the masking step skipped. This part of the code is not covered by the unit tests, as it involves hard to test iris methods.
If it's viable, I'd be in favour of using a slightly bigger file which requires more of the code to run. The whole aiihca.paa1jan file that aiihca.paa1jan.subset is derived from is 511M, perhaps too big. I'm wondering whether there'd be a possible middle ground, and whether the choice of file determines where the CI tests should run?

I think the main objective of the regression tests is to make sure the code functionality is as expected.
These are especially important when unit-testing is not complete (or not present at all).
I think in general we should perform a bunch of tests that verify different options/entry-points based on the specific project.
For this project I think it's important to test the heaviside masking, along with the conversion_driver_esm1p5.py script.
Given the rather quick elapsed time for the tests (around 15s from the output you shared), we could perform 3-4 different tests for the um2netcdf.py and maybe 1-2 tests for the conversion_driver_esm1p5.py. This way we could test different things, still keeping the tests to a total time ~2m.
For the fieldsfile to use, I think the best option is to prepare "ad hoc" fieldsfile (unless we already have suitable ones) that are not too big (I would try not to go more than 20M each) and that are suitable to test the particular options we want to test (for example the heaviside masking).


Add examples of Select using a slice of q.Matcher

I wanted to add some examples showing how conditionally someone could append various q.Matchers to a filter slice and showing you that err is thrown when no data is found on a Find.  This is a helpful example to see how to compose a query where logic in your algorithm will append filters based on various conditions since technically it seems like you can only call Select once to build the And's and cannot modify it after.  The slice example is how I am going to need implement things because not all my callers will know 100% the full filter when it's executed so it needs to be built up over time and then the slice of q.Matcher is passed to Select().
In this case of the data2 block, if I merely re-used data, the records would still exist because when storm cant find anything, it keeps your referenced data in there and doesnt clear it.  So anyone using this, make sure you handle errors like I did with the "not found" check so that you typically wont want to return error back on a not found when filtering data, rather just pass back the empty slice of data back to your caller who is making the query.

very cool, thank you!

Your examples thing is #2 hit on google when you search storm examples boltdb
I was able to play with your example to figure out how to compose slices of queries to pass to the Select function of storm.
https://github.com/DanielRenne/GoCore/blob/master/core/dbServices/bolt/stubs/query#L1489-L1591
Basically we wrote a bunch of crazy mongo queries with mgo with our own little DSL and now it is starting to support some storm/boltdb, so that is exciting so we can get off mongo for smaller computers where mongo doesnt shine for simple application deployments on simple processors.
We basically generate models based off a JSON schema definition and this is one of the stubs
https://github.com/DanielRenne/GoCore/blob/master/core/dbServices/mongo/stubs/query
If you want to spawn a goCore app you would need npm installed and node but then do a
go run core/goCoreCreateApp/main.go
None of this stuff is fully documented yet, but you would login as admin/admin for your initial dev user.  Let me know if you play with it.  Storm can login and set password, but im still working on some more complicated And/Or queries which are busted converting from the mgo world into bolt.

You can just read the source code of your spawned app to see how we setup models in gocore.  They are in db/schemas/ and some db/bootstrap/ data as to how we parse the schema format to allow you to generate fields in bolt or mongo for []string and objectArrays etc and have a model system setup to grab data.  We have some really sick react and websocket stuff going on with our "store" example in serverSettings utilizing some reliable websocket code we wrote based off of gorilla websockets.


feat: add icons and enablement conditions for commands

feat: add icons and enablement conditions for commands
This commit adds icons and "when" enablement conditions for various Cody commands in the package.json file.
Test plan

Commands should be toggled based on the new when clauses.

Looks good but editorTextFocus is misspelled editorTextFocusd a few places.

my brain wasn't working yesterday 🤡  Thanks for catching them!


Document references VSTS

In a few places in the document it still references VSTS.  It should reference Azure DevOps

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 44afc4fe-66c8-a745-0037-00748a80beb7
Version Independent ID: 0ea09401-6a08-d745-ff50-7a7b74c41f32
Content: Migrate from TFVC to Git - Azure DevOps
Content Source: https://mseng.visualstudio.com/DefaultCollection/TechnicalContent/_git/devops-learn-pr?path=%2fdocs%2fgit%2fmigrate-from-tfvc-to-git.md&version=GBlive&_a=contents
Product: devops
Git Login: samgu
Microsoft Alias: routlaw

@jamers99 Thanks for letting us know, I will get this fixed up today.

@steved0x do we need to update Team Foundation references with Azure DevOps Server released?


woodpecker: 0.15.5 -> 0.15.6

Description of changes
https://github.com/woodpecker-ci/woodpecker/releases/tag/v0.15.6
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

@ofborg eval

@ofborg build woodpecker-cli woodpecker-server woodpecker-agent

The system CA error patch can be dropped now, as it's made it into this release.
Please test to make sure things build before PRing, as this could have been easily caught given a test build. Thanks!

(Same goes for you @ambroisie, I'd recommend not approving PRs until you've tested them. Thanks!)

I made the change, tested that it builds, and force-pushed so we can land this.
@ofborg build woodpecker-cli woodpecker-server woodpecker-agent

Thanks so much @winterqt


add initial CONTRIBUTING.md

Can we release under the MIT license when statically linking an LGPLv3 library?

Can we release under the MIT license when statically linking an LGPLv3 library?

it's fine, our legal had gone over the licenses


the .layout and .depends file should not be put in the git repo

Hi,
.layout stores the window layout, .depends stores the build dependencies. Those files are automatically generated by Code::Blocks when running or compiling.
Generally, a single .cbp file is enough for a Code::Blocks project.
The issue also exits in your another repo: https://github.com/yan9a/ceutil

Thanks for the feedback. I will remove them and get back to the issue to close.

Removed and added gitignore

There is one layout file in wxtest folder.

Thanks again. wxtest.layout now removed.


the command (npm test ) does not work after the command (npm install)

after npm install , warn :  peerDependencies WARNING<EMAIL_ADDRESS>›<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but<EMAIL_ADDRESS>was installed
after npm test , warn :
Cannot find module 'babel-plugin-annotate-pure-calls' from
if i switch<EMAIL_ADDRESS>into<EMAIL_ADDRESS> there is still a error

This happens after you clone the repository, right? Not when installing the package from npm

I guess, the babel-plugin-annotate-pure-calls is installed not via npm repository .
But still there is another mistake and I think, it is connected to babel-plugin-annotate-pure-calls
> cross-env BABEL_ENV=es babel src --out-dir es

src/index.js → dist/redux-saga.min.js...--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

src/index.js → dist/redux-saga.js...
src\effects.js -> es\effects.js
[!] (babel plugin) TypeError: Cannot read property 'isProgram' of null
src\index.js
TypeError: Cannot read property 'isProgram' of null
    at error (C:\Users\las\Desktop\Projects\redux-saga\node_modules\rollup\dist\rollup.js:185:14)
    at Object.error (C:\Users\las\Desktop\Projects\redux-saga\node_modules\rollup\dist\rollup.js:9469:6)
    at promise.then.previous (C:\Users\las\Desktop\Projects\redux-saga\node_modules\rollup\dist\rollup.js:9478:32)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
    at Function.Module.runMain (module.js:667:11)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:607:3

TypeError: Cannot read property 'isProgram' of null
    at isTopLevel (C:\Users\las\Desktop\Projects\redux-saga\node_modules\babel-plugin-annotate-pure-calls\lib\index.js:58:34)
    at isExecutedDuringInitialization (C:\Users\las\Desktop\Projects\redux-saga\node_modules\babel-plugin-annotate-pure-calls\lib\index.js:62:7)
    at PluginPass.callableExpressionVisitor (C:\Users\las\Desktop\Projects\redux-saga\node_modules\babel-plugin-annotate-pure-calls\lib\index.js:103:8)
    at newFn (C:\Users\las\Desktop\Projects\redux-saga\node_modules\@babel\traverse\lib\visitors.js:223:21)
    at NodePath._call (C:\Users\las\Desktop\Projects\redux-saga\node_modules\@babel\traverse\lib\path\context.js:64:19)
    at NodePath.call (C:\Users\las\Desktop\Projects\redux-saga\node_modules\@babel\traverse\lib\path\context.js:38:17)
    at NodePath.visit (C:\Users\las\Desktop\Projects\redux-saga\node_modules\@babel\traverse\lib\path\context.js:95:12)
    at TraversalContext.visitQueue (C:\Users\las\Desktop\Projects\redux-saga\node_modules\@babel\traverse\lib\context.js:139:18)
    at TraversalContext.visitSingle (C:\Users\las\Desktop\Projects\redux-saga\node_modules\@babel\traverse\lib\context.js:98:19)
    at TraversalContext.visit (C:\Users\las\Desktop\Projects\redux-saga\node_modules\@babel\traverse\lib\context.js:180:19)
src\index.js -> es\index.js

The problem is definitely in babel-plugin-annotate-pure-calls.

Truth to be told - I have no idea what's the problem. I've done fresh installs on my machine and they are working fine. Could you describe exact steps that led to this problem? Also specifying node/npm versions would be helpful.

It seems like I got the idea - there was a breaking change in Babel itself.
And that line uses the old version.
P.S.
npm - 3.10.8
node - 8.6.0

For some reason your npm has downloaded master branch from babel-plugin-annotate-pure-calls instead of the specified babel7.

Hmm... The problem was in an old version of npm, but yarn doesn't work neither.
By the way, we can add this field in order to prevent developer mistakes.

I'll release the babel plugin later under other tag than latest, should fix problems like this.

I've published the babel plugin to npm and changed redux-saga's dep on it. Things should work for you guys now, let me know if all is ok.
@Arhane unfortunately engines field specifies which versions of i.e. npm are capable of installing/using the package as user, not as developer of the package and there is no devEngines field :(


Transparency debt

I'm raising an issue about this in general because I see it as a pattern we should be aware of. We may need specific issues for different parts of it.
I have noticed multiple examples of what I'm calling "Transparency Debt" in our system, where through a series of changes or decisions we've strayed away from our commitment to transparency and things are getting hidden on the platform that shouldn't be.
Hidden Budgets
Here's a data query showing profiles with hidden budgets.
When Funds were created, we did not yet have #3689 and decided to allow hiding budgets due to specific valid privacy requirements around expenses from some specific Funds. However, now that we have the ability to use pseudonyms on expenses, this is no longer necessary. I have discussed this with @piamancini and I will get in touch with the Funds who have hidden budgets and help them to transition.
@Betree has already turned off the ability for Funds to switch off their budgets going forward. We can take requests via support to hide budgets, but I think our policy should be not to do so and instead push users to use the pseudonym function if they're concerned about expense payee identities being public.
 I think we should treat this as a bug, inform the users, and show the Project budgets.
Host hiding Collectives
At the moment, WWCODE is not displaying their hosted Collectives. This is possible because the "contributions" section can be hidden, and hosted Collectives show up within that section on Host pages. Personally I think that hosted Collectives should be its own section for a variety of reasons, but one way or another we should prevent hosts from hiding this information.
User hiding expenses
Related to #5261 - while you can see a user's expenses on the Collective's expenses list, the user can hide them on their own profile. This makes it hard to see which Collectives a given user is getting paid from and to transparently understand their activities. We have aspirations to build #3749 to create a proper expenses page per user, and it's odd that this information can be hidden currently.
Are there any other examples of transparency debt we should know about?

The "hidden budget" part will be resolved in https://github.com/opencollective/opencollective-frontend/pull/7532 for new accounts (regardless of their types - projects, funds, etc.) as we're not letting them hide their budget sections anymore unless they're already hidden (which means the engineering team can still manually hide them if needed).

I think closing was a mistake with automation, reopening.

I think closing was a mistake with automation, reopening.

It was, thanks for re-opening. I fixed the description in https://github.com/opencollective/opencollective-frontend/pull/7532

After discussing with team members, I sent a message to the admins of all Projects with hidden budgets that are not part of Funds, as we consider Collective Projects hiding budgets to be a bug. I've asked them to let me know in the next week if they object to their budgets being displayed.

Just thought of another transparency thing... in #4640 we decided to hide archived projects. This means the transactions of past Projects will not be visible, which is not good for transparency.
Since the main purpose of hiding archived Projects was to prevent confusion by users about what's currently active, I wonder if we could create a little link to "see archived projects" which goes to a separate page, so if people are looking for a transparent history, it's findable, without making people think inactive Projects are still active.

I wonder if we could create a little link to "see archived projects" which goes to a separate page, so if people are looking for a transparent history, it's findable, without making people think inactive Projects are still active.

@alanna That's already more or less the case. On the collective page, you have this:

When clicked, the projects page has everything listed:

I meant one more click, so on the /projects page there would be a "see archived projects" button. This is because Collectives who use Projects a lot told us having archived projects there was confusing.

just chiming in to say that i am today writing to confirm that we won't mark contributions as incognito unless the request comes from the contributor themselves and that this is a policy decision that needs to be reflected somewhere alongside the decisions we're making in this thread.
Perhaps this should feed into the ethical standards conversation that @natehn is leading on https://github.com/opencollective/opencollective/issues/4928

@BenJam I don't think that those sorts of policies would live in the community guidelines - more of a documentation thing, I think. But in general, yes, this is interconnected with the discussion of Privacy and Transparency we have in the drafted Community Guidelines

As mentioned above, we decided to treat hidden Project budgets not under Funds as a bug and un-hide them. I informed all the admins of such projects a week ago to let me know if they have any objections, and no one did. So I have raised this FD ticket to ask @znarf to do this.
Please note, there are a bunch of Projects with hidden budgets not on this list under the NumFOCUS host. I am in touch with them about that separately (they aren't against transparency they just have many questions about how to set stuff up).

Update:

Projects not under Funds have been un-hidden.
New Funds and Projects cannot hide budgets themselves (we will decide what to do if someone requests a hidden budget in the future, but at least we will know about it)

What's the rationale behind allowing Collectives to hide their admins (by hiding the team section in settings)? I think this is a big issue for transparency.


Auto-start custom battle does not work, WH3

I have set up battle so only thing I need to do after starting game is to open custom battle and start it, and ticked Auto-start custom battle after that yet nothing happens after on next game start, its like I have not ticked that option.

Tried that and it works, thanks (was using skip intro mod before)


Remove script generating lot and declaration questions

After heavily refactoring this script, I moved it to the digitalmarketplace-scripts repository.
Just coming back to do some cleanup.
>> Relevant pull request

Easy :+1:


NamespacesAnalyzer - Optimize performance

Instead of looping on all tokens, even inside the namespace it identified, the analyzer now continues the analysis after the end of the identified namespace, thanks to the fact that namespaces cannot be nested.
When implementing #3876, I compared the NamespacesAnalyzer implementation with \PhpCsFixer\Fixer\FunctionNotation\NativeFunctionInvocationFixer::getUserDefinedNamespaces. The NamespacesAnalyzer also identifies the namespace name instead of skipping over it (and returns the global namespace too). But I found out that NativeFunctionInvocationFixer was more optimized, as it was not re-analyzing all tokens inside a namespace. This makes NamespacesAnalyzer implement the same optimization.

Thank you @stof.

We do push code refactoring to oldest maintained branch,

I was not aware of that. Good to know.

for next MINOR branch we do push only what MINOR is reserved for - features ;)
all the rest, like bugs, docs, typos, refactoring - to lowest possible branch.


add geometries functions

I've been having issues with the CI actions, could you please rebase this PR against the main branch please

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

36 of 36 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.006%) to 99.436%

Totals

Change from base Build<PHONE_NUMBER>:
0.006%

Covered Lines:
3176

Relevant Lines:
3194

💛  - Coveralls


Generate GBL endpoint and backend

Description
This PR adds an endpoint to the public API that takes a shipment id, creates a pdf of the GBL with data for that shipment and adds it to the move documents as a GBL document. If there is already a GBL for this document, then calls to create a new one result in an error (400). If there is missing data, a 417 is returned. If the creation is successful, an Upload payload is returned in the response (with URL to the pdf doc), but the client is not currently doing anything with it.
It also adds a move document type for a "GOV_BILL_OF_LADING" in the PUBLIC swagger (api.yaml), but not in the private yaml. It is added to the move document model as well in order to be referenced, but currently the move document type generated to the api messages package via swagger is not being used to validate the type.
To manage private static assets, this PR also uses a Docker image that has go-bindata installed  (https://github.com/kevinburke/go-bindata)(https://github.com/trussworks/circleci-docker-primary/pull/6). go-bindata allows us to compile assets into the Go build, and then access the byte arrays via an "Assets" package. The command added to the makefile, server_go_bindata, checks if there are any new assets at pkg/paperwork/formtemplates/, and if so, generates the assets package at pkg/assets/assets.go with the added files.
TODO:

update bin/prereqs to install go-bindata for local development
add ADR for process to add static assets to be compiled into go binary?

Reviewer Notes

The upload payload definition is copied from internal.yaml over to the public api.yaml - this is not ideal and would be commented if not in a yaml file, but a solution for this current PR. When the Document viewer is implemented in the TSP app and doc payloads are defined, this handler should return that instead.
We will eventually want to be able to return more informative error messages for what required GBL data does not exist - currently, there's nothing more granular than a 417.

Setup

Install go-bindata- brew install go-bindata. (note: if you install via go get -u github.com/kevinburke/go-bindata/..., you may run into issues of go upgrading beyond 1.10 (the version we are currently pinned to).

make db_dev_migrate to add shipment ID field to the move document table, and make db_populate_e2e.
Through swagger UI: with no move docs of type GOV_BILL_OF_LADING in your db, In the swagger ui for the TSP app (http://tsplocal:3000/api/v1/docs), pass in a shipment ID into the gov_bill_of_lading endpoint. You should get back a 200, and have a new GBL move doc associated to that shipment.
Through the TSP UI: login as a TSP user and navigate to BACON2 (move locator) Shipment info page. You should see a button on the right, "Generate Bill of Lading" - click it the first time and you should see a success message (see image below). Click it another time and you should see an error message (see image below).

Code Review Verification

[x] End to end tests pass (make e2e_test).
[x] Code follows the guidelines for Logging
[x] The requirements listed in
Querying the Database Safely
have been satisfied.
[x] Any migrations/schema changes also update the diagram in docs/schema/dp3.sqs
[ ] There are no aXe warnings for UI.
[ ] This works in IE.
Any new client dependencies (Google Analytics, hosted libraries, CDNs, etc) have been:

[ ] Communicated to @willowbl00
[ ] Added to the list of network dependencies

[x] Request review from a member of a different team.
[x] Have the Pivotal acceptance criteria been met for this change?

References

Pivotal story for this change

Screenshot

I talked with Nick and we're going to go with this data model change in your PR for move_documents.  I'll update my other PR #976 after you merge this to follow on from your work.


update alias for dfvindex.eth

Hi! What is your PR about?

[] Add images for your space
[ ] Add custom domain for your space
[x ] Add alias for migrate your space
[ ] Add or edit a skin
~Edit your space with GitHub?~ (deprecated) Please migrate your space to ENS https://docs.snapshot.page/spaces/migrate
~Create your space with GitHub~ (deprecated) Create your space with ENS instead https://docs.snapshot.page/spaces/create

Thank you for reviewing.
I had added a pull request for a new skin yesterday in the dfv.css file under skins. (which was approved).
Yet when I go to my settings on https://snapshot.org/#/dfvindex.eth/settings and click add skin I receive the "oops, we can't find any results prompt."
On Discord, it looked like the way to solve this was to add an alias, but clearly I have not followed the guidance correctly per the above.
Any thoughts or feedback on how I can resolve this issue?
Thank you for the consideration.


Add support to letsencrypt's' EC certificates

Use generic openssl pkey instead of openssl rsa and compare the hashes of the private key's public key.

Use generic openssl pkey instead of openssl rsa and compare the hashes of the private key's public key.

oui

Use generic openssl pkey instead of openssl rsa and compare the hashes of the private key's public key.

oui


How to change the selection in DD-MM-YY format instead of MM-DD-YY

Hi need help to alter the selection FORMAT to DD-MM-YY  from showing MM-DD-YY ..how to do this ??

I would also like to know this :)

Shouldn't the plugin read in the locale and be able to print out locale specific format?


Expose Debug log level for nakama applications

It would be nice to expose the Debug log level.  Currently the log levels that are exposed to a nakama application are INFO,WARN, and ERROR.  Nakama internally uses DEBUG for various things, it would be nice to expose this at the application level as well.

Currently the log levels that are exposed to a nakama application are INFO,WARN, and ERROR.

@jordo Where did you read that information from? (It would be good to clear it up in the documentation if needed.)
We already support ERROR, WARN, INFO, and DEBUG levels (though we don't have a TRACE level yet). This can be configured with --logger.level DEBUG in cmdline arguments or via YML:
logger:
  level: DEBUG

We also have functions to write to all logger levels via the server framework. i.e. logger_debug() in Lua, logger.debug() in JavaScript, and logger.Debug() in Go.

@novabyte Ah yes, this is just a documentation issue:   https://heroiclabs.com/docs/nakama/server-framework/function-reference/#logger


hostPort is not working after upgrade from calico v3.15.5 to any latest(v3.16.0 above)

Expected Behavior
In k8s deployment container hostPort should expose to the host level
Current Behavior
with calico version v3.16.0 and above hostPort functionality broken
Possible Solution

Steps to Reproduce (for bugs)

Create a Kubernetes cluster with version v1.19.8 and use Calico CNI v3.16.10
Deploy an application with exposing SNMP trap port 162 as hostPort
Check that port is exposed on the host level or not.
With the same Kubernetes v1.19.8 and Calico v3.15.5, when we expose container port as hostPort then on host also listening on same ports.

Context
Due to this issue applications will not support SNMP traps, Syslog, telemetry functionalities
Your Environment

Calico version v3.16.10
Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes v1.19.8
Operating System and version: Centos7 with firewalld enabled
Link to your project (optional):

Hey,
We get the same issue while using firewalld on those kubernetes servers.
We run Kubernetes v1.16 via kubeadm on prem, and we are trying to upgrade from calico 3.3.x to 3.16.x .
The calico upgrade is affecting the chains order in the iptables in a way that blocks traffic to pods from outside the node subnet. It is happening because we are using REJECT first rule in firewalld.
We solved it by changing the firewalld default zone rule to 'default' instead of REJECT. after that it seems to work again and also firewalld is still working properly.

@caseydavenport Is this still an active problem?  I am experiencing a problem with Datadog APM not working because the hostPort isn't being mapped in correctly.  We are running a cluster that is using version 3.18.1 of calico...
This is in our configmap so it should be working...
        {           "type": "portmap",           "snat": true,           "capabilities": {"portMappings": true}         },

We get the same issue while using firewalld on those kubernetes servers.

This sounds different, and is clearly called out in the docs requirements for Calico:

If your Linux distribution comes with installed Firewalld or another iptables manager it should be disabled. These may interfere with rules added by Calico and result in unexpected behavior.

@caseydavenport No firewalld enabled in my environment that I'm aware of.  3.18.1 is definitely still having this problem for us.

We run e2e tests that verify host ports function properly regularly, so I can confirm that in some capacity hostPorts should be functioning. There may be something specific about your environment that's causing this.
host ports are implemented in iptables by the portmap plugin - can you confirm whether or not you see those rules being programmed in iptables for pods with hostPort set?

I asked our managed provider to discuss upgrading calico to the current version.  We use platform9 as our provider.  Worked for quite a number of days with Datadog engineering to come to the conclusion that likely the version of calico we are on or some other setting is causing it to not map hostPort correctly.  When we look it does a mapping to a dynamic port and the host never actually listens on the hostPort that is required...  Any guidance on troubleshooting the issue from calico's perspective to gain some insight into it?  Like look for logs somewhere etc?

Not sure if this is relevant, but we have first calico, then portmap in the cni config, and we set externalSetMarkChain explicitly:
      "plugins": [
        {
          "type": "calico",
.....
        },
        {
          "type": "portmap",
          "snat": true,
          "externalSetMarkChain": "KUBE-MARK-MASQ",
          "capabilities": {"portMappings": true}
        }
      ]

When we look it does a mapping to a dynamic port and the host never actually listens on the hostPort that is required... Any guidance on troubleshooting the issue from calico's perspective to gain some insight into it? Like look for logs somewhere etc?

The most simple thing, like I mentioned above, is to confirm whether or not the iptables rules are being programmed by the portmap plugin and if so, are they being hit?
sudo iptables-save -c

^ This should tell you whether the rules exist, and give packet/byte counts to see if the rules are being hit. Note that if something else on your host is bound to the hostPort, I believe it will take precedence and traffic will be consumed by the process bound to the port. You should never use a hostPort that matches a port already in use on the node. In generally, I highly recommend against using host ports as they are a tricky and contentious host resource to manage.

Thanks @caseydavenport if I had another choice for a kubernetes deployment of Datadog agent I would use that.  Their APM uses hostPort to bind to 8126 and so that's what I got.  I've worked around it for the time being with a service but that messes up the metric correlation as it might be from any machine since it's now using a service instead of hostPort.
I got a lot back from that iptables-save if it's working you think I should see something about port 8126 in the output?

[0:0] -A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: "k8s-pod-network" id: "a05ea9f3874b0d9471941ddaa3253c437a07a1e7031d1e427b8a21f21383b525"" -m multiport --dports 8126 -j CNI-DN-ddb0866915bc323564499

Doesn't look like any packets are hitting that rule. So, either there was no traffic to the pod around the time that you ran this command, or there is another rule earlier in the chain that is accepting / denying the traffic before it reaches here.
I'd recommend trying to ensure there is traffic flowing to the service on the hostport, and then monitor to see if it is being handled by another rule. From the netstat output, doesn't look like anyone else is listening on that port so that's good.


Run rails db:create in Dockerfile. Closes #6

@zachlatta Not sure if it's the right way, but it sounds like #6 should be fixed by a change to the dockerfile.

This isn't the solution because the Dockerfile is for the Rails app – rails
db:create touches the database and there are no guarantees for how that's
set up.
This'll break production if merged, closing.
On Thu, Jul 28, 2016 at 8:46 PM, Max Wofford<EMAIL_ADDRESS>wrote:

@zachlatta https://github.com/zachlatta Not sure if it's the right way,
but it sounds like #6 https://github.com/hackclub/lecture-hall/issues/6
should be fixed by a change to the dockerfile.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hackclub/lecture-hall/pull/8#issuecomment-236107099,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA8j-Lvl9Uxt73jVVsCOF8Ssvkol59Jxks5qaaG_gaJpZM4JX63I
.

--
Zach Latta
https://zachlatta.com http://zachlatta.com


Minor changes to various class behaviors

Small changes to how the device and memory manager class work
In some more interesting test cases of the device and memory manager classes, I found that the device/memory manager classes didn't behave how I'd expect.
These small changes make sure things behave in an expected ways. For example, if a device object is dropped / deconstructed the internal singleton execution strategy would be reset. We don't want this as it would affect how later calls might behave.
Next, the memory manager would not set the initial execution space to be something consistent with what the device object is using. This rectifies that difference. Additionally, it fixes a small bug in the pointer version of the chai managed array function where the current execution space isn't used in the creation of the object.
Fix how failures in linear solve are handled by nl solvers
In our nonlinear solvers, we would often SNLS_FAIL out if our linear solve for the newton step failed. This behavior is unwanted in a number of situations. For example on GPU runs, we would only print an error and continue on happily without external codes being aware an issue has occurred. In implicit FEM codes, we can encounter these sorts of issues from a large time step. Rather than having a code crash because it didn't catch the exception, a failed status of the nonlinear solver is more graceful and allows the code decide how best to handle things such as by cutting the time step.
These changes therefore change these fails to warns and modify the nonlinear solve to return a failed status from which application codes can more gracefully recover from.

@nrbarton let me know if you have any thoughts on this as I'd like to include this in a few library/application updates.

The changes look fine overall. As far as error reporting, passing an error status up the call stack is fine if the call stack is not too deep. It could get awkward if the call stack were much deeper though. So in other settings it might be nice to rework SNLS_FAIL or the equivalent to set an error code for the thread and suspend execution of the thread so that it does not continue to churn if it is in a loop or the like ... but I do not know if there is a nice mechanism to support such a suspension of  the thread with an error.

@nrbarton yes I believe what you're talking about would make more sense for whatever library/application code is using SNLS. I've got a branch of ExaCMech somewhere that actually goes through how to deal with that sort of thing on the GPU/CPU/OpenMP.
Sadly, C++ doesn't really have a nice native error handling strategy that just lets us bubble these sorts of things effortlessly up the call stack.


Handling long lists

I am using awesomplete in a tool I built to list assets to pick. But the problem is the list is so long that it runs off the page. Even when you filter, there are enough items starting with the same letters to run off the page. I was wondering if it’s  possible to have the results of a long list show in columns.

This isn't really something that can be handled with Awesomplete itself, but you could probably get there with CSS, setting the list max-height to something like 80vh so it's guaranteed to not be taller than than the screen. You could then use css columns, although you'd probably want to have a different number of columns based on viewport width and number of results (if you only have 4 results, it may be odd for each to be in a different column).  It may be easier to set the results overflow-y: scroll, so you can scroll within the long list of results.


Enhancement: Include $lower attribute for dashed

By including $lower in the attributes for dashed it allows for the use cases of generating gift card codes or serial numbers (generally uppercase only) in a more efficient way than using strtoupper afterward.

Nice, I really like this!
Would an option to support upper be useful here as well? I guess you can apply upper on the output, but having it as part of it could be nice.

Ah good point, I misunderstood what it did, so the naming of $lower clearly doesn't work.
I like the idea of using $mixedCase instead - it's clearer what's going on.


New MissingRuntimeArtifactException in Release mode after EF Core 2.0 upgrade

We upgraded:

UWP 10.0.15063.0 to 10.0.16299.0
Microsoft.EntityFrameworkCore.Sqlite 1.1.3 to 2.0.0
Net Standard 1.4 to 2.0.1

We now get the below error when the app runs in release mode (w/.net native compilation) with various queries:

System.Reflection.MissingRuntimeArtifactException occurred
  HResult=0x8013151A
  Message=Cannot retrieve a MethodInfo for this delegate because the method it targeted was not enabled for metadata using the Dynamic attribute. For more information, please visit http://go.microsoft.com/fwlink/?LinkID=616868
  Source=
  StackTrace:

Sample query where the error happens:
public async Task<IList<ServingOptionCategory>> GetServingOptionCategories()
{
    IList<ServingOptionCategory> servingOptionCategories =
        await _menuContext
            .ServingOptionCategories
            .Include(soc => soc.ServingOptionCategoryTag)
            .AsNoTracking()
            .ToListAsync();

    return servingOptionCategories;
}

This behavior didn't happen before targeting the previous versions.
Steps to reproduce
See the repo: https://github.com/thnk2wn/UwpReleaseBuildEfCoreIssue
Further technical details
EF Core version: 2.0.0
Database Provider: Microsoft.EntityFrameworkCore.Sqlite
Operating system: Win 10 Enterprise, Ver 1709, OS Build 16299.19
IDE: Visual Studio Enterprise 2017 15.4.1
Expected Behavior
EF Core query runs the same without error in Release mode as it does in Debug. This was the behavior when targeting Microsoft.EntityFrameworkCore.Sqlite 1.1.3, .NET Standard 1.4, and UWP 10.0.15063.0.

@thnk2wn Could you try with the 2.0.1 pre-release bits as described at the end of this issue: #9355? We fixed a few issues with .NET Native in the 2.0.1 release, so it would be good to know if this is one of them.

Thanks @ajcvickers that appears to fix this issue.
These might be separate (but perhaps related) issues but I could report them separately later if needed.
There's a new 'hidden' exception calling Context.Database.EnsureCreatedAsync:

Microsoft.Data.Sqlite.SqliteException occurred
  HResult=0x80004005
  Message=SQLite Error 14: 'unable to open database file'.
  Source=
  StackTrace:

Performance is worse. Rebuilding our DB went from 2 minutes to 3 minutes in Debug builds and is 5 minutes in Release builds.

Unable to open database could be related to https://github.com/aspnet/Microsoft.Data.Sqlite/issues/408

@thnk2wn Can you be more specific about what, "rebuilding our DB" means? Is this just building the EF model, or is it also running migrations, doing EnsureCreated, etc.?
I'm closing this as a dupe since it looks like both the original issue and the SQLite issue are already tracked.
Duplicate of #9355

Sorry @ajcvickers. By rebuilding our DB I'm referring to populating some 47 tables with data that comes from web services. That time excludes creating the DB itself and the web service calls - it's just taking the deserialized data, doing a little mapping/projecting in cases, and doing AddRangeAsync and SaveChangesAsync.
Are we better off excluding our EF project from .net native compilation? I see various issues on this topic such as https://github.com/aspnet/EntityFrameworkCore/issues/5570


Field syntax for AppliedLens

Partially implement #735
case class Foo(i: Int, bar: Bar)
case class Bar(b: Boolean, s: String)
val foo = Foo(5, Bar(true, "Hello"))

foo.optic.field(_.bar).field(_.b).get shouldEqual foo.bar.b

We should also be able to chain fields like _.bar.b but there is currently a compiler error.
We can repeat the process for all optics, applied optics.

Maybe I can try to do the same for another of the optics, Prism perhaps?

Maybe I can try to do the same for another of the optics, Prism perhaps?

Sure. Do you mean add field to Prism / AppliedPrism or add another method on AppliedLens that simplifies composing with the branch of a sealed trait?

I know a bit about macros and this looks reasonable to me 😄

Thanks @cb372 for the review. Do you have an idea of how we can generalise that macro to other optics/applied optics without completely duplicating the code?

Yes that should be do-able, but I'll have to have a look at the rest of the code. I haven't been following the progress of 3.x so I need to catch up. Is there any kind of design doc I should look at before I dive into the code?

It is not really formalised. We have a couple of ideas but nothing really set in stone. This is probably the best plan we have so far: https://github.com/julien-truffaut/Monocle/issues/714.
Regarding this particular point, we are trying to have a convenient syntax to compose optics for the most common scenarios such as a user can do:
someOptioc.field(_.foo)

instead of this syntax in Monocle 2:
someOptic composeLens GenLens[Bar](_.foo)

We are trying to implement field this method on all optics (Lens, Prism, Iso, ...) and all applied optics (AppliedLens, AppliedPrism, AppliedIso).
I am not sure where these methods should be defined. Should we have them on optics/applied optics classes or should we "pimp" them on a separate module.


TP2000-888  Add ME40 validation to create a new measure journey

TP2000-888  Add ME40 validation to create a new measure journey
Why
Create a new measure journey lacks validation to prevent ME40 violations.
What

Fixes the displaying of duties under conditions on the review page
Commodities and conditions forms check if duties are permitted for the selected measure type
Review form checks if mandatory duties are missing for the selected measure type
Updates and adds tests

Codecov Report
Patch coverage: 100.00% and project coverage change: +0.01 :tada:

Comparison is base (c86a7d7) 92.53% compared to head (43c3a19) 92.55%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #934      +/-   ##
==========================================
+ Coverage   92.53%   92.55%   +0.01%
==========================================
  Files         431      431
  Lines       30063    30115      +52
  Branches     2292     2300       +8
==========================================
+ Hits        27820    27872      +52
  Misses       1788     1788
  Partials      455      455

Impacted Files
Coverage Δ

measures/forms.py
95.66% <100.00%> (+0.11%)
:arrow_up:

measures/tests/test_forms.py
99.59% <100.00%> (+0.01%)
:arrow_up:

measures/tests/test_views.py
100.00% <100.00%> (ø)

measures/validators.py
96.66% <100.00%> (+0.20%)
:arrow_up:

measures/views.py
97.52% <100.00%> (+0.03%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

I've just realised we don't do this for any of the measures views but on some other models (geo areas, footnotes etc) we use views that inherit from TrackedModelChangeView which uses the BusinessRulesMixin allowing us to set validate_business_rules property on the class with a list of business rules to check against on form submission. I haven't investigated this myself but would it work to use this same pattern here? Or if it doesn't work with the form wizard, maybe we should find a way to rewrite the mixin in a way that does work. Then we could just reference the business rule instead of pulling out the logic that lives inside it

That’s an interesting idea and would certainly be preferable. To make use of the mixin would likely require some reworking in terms of passing the object (new measure) to be checked and making violation error messaging user friendly. But it sounds like it’s worth creating a follow-up ticket to investigate further.