[Bug] pd.DataFrames modified during ingest/outgest round-trips

(Factored out of #2804)
Describe the bug
Certain pd.DataFrame column/index names can be shuffled or dropped during an ingest/outgest round-trip (e.g. {from,to}_anndata). In some cases, a column or df.index can be dropped.
To Reproduce
See test_dataframe_io_roundtrips.py.
Versions (please complete the following information):

TileDB-SOMA version: 1.12.3, seems like ingest/outgest have always worked this way though
Language and language version: Python, all supported versions
OS: all

Examples
(see test_dataframe_io_roundtrips.py; obs-roundtrip.py does similar and is used in the code snippets below)

1. df.index is named "index":

df.index.name is dropped (index becomes unnamed)

python obs-roundtrip.py -i index=xx,yy,zz -c col0=aa,bb,cc -c col1=AA,BB,CC 2>/dev/null
# Before:
#       col0 col1
# index
# xx      aa   AA
# yy      bb   BB
# zz      cc   CC
#
# SOMA DataFrame schema:
# soma_joinid: int64 not null
# obs_id: large_string
# col0: large_string
# col1: large_string
# metadata[soma_dataframe_original_index_name]:
# null
#
# After:
#    col0 col1
# xx   aa   AA
# yy   bb   BB
# zz   cc   CC

2. DataFrame has a column named obs_id:

obs_id column becomes unnamed index
Original df.index dropped

python obs-roundtrip.py -i xx,yy,zz -c col0=aa,bb,cc -c obs_id=AA,BB,CC 2>/dev/null
# Before:
#    col0 obs_id
# xx   aa     AA
# yy   bb     BB
# zz   cc     CC
#
# SOMA DataFrame schema:
# soma_joinid: int64 not null
# col0: large_string
# obs_id: large_string
# metadata[soma_dataframe_original_index_name]:
# null
#
# After:
#    col0
# AA   aa
# BB   bb
# CC   cc

DataFrame has a column named "index"

3. df.index is unnamed:

"index" column is promoted to df.index (unnamed)
Original (unnamed) index becomes a column named level_0

python obs-roundtrip.py -i xx,yy,zz -c col0=aa,bb,cc -c index=AA,BB,CC 2>/dev/null
# Before:
#    col0 index
# xx   aa    AA
# yy   bb    BB
# zz   cc    CC
#
# SOMA DataFrame schema:
# soma_joinid: int64 not null
# level_0: large_string
# col0: large_string
# obs_id: large_string
# metadata[soma_dataframe_original_index_name]:
# null
#
# After:
#    level_0 col0
# AA      xx   aa
# BB      yy   bb
# CC      zz   cc

4. df.index is named id_column_name (default obs_id):

"index" column is dropped

python obs-roundtrip.py -i obs_id=xx,yy,zz -c col0=aa,bb,cc -c index=AA,BB,CC 2>/dev/null
# Before:
#        col0 index
# obs_id
# xx       aa    AA
# yy       bb    BB
# zz       cc    CC
#
# SOMA DataFrame schema:
# soma_joinid: int64 not null
# obs_id: large_string
# col0: large_string
# metadata[soma_dataframe_original_index_name]:
# "obs_id"
#
# After:
#        col0
# obs_id
# xx       aa
# yy       bb
# zz       cc

5. df.index has another name:

"index" column renamed to obs_id

python obs-roundtrip.py -i idx=xx,yy,zz -c col0=aa,bb,cc -c index=AA,BB,CC 2>/dev/null
# Before:
#     col0 index
# idx
# xx    aa    AA
# yy    bb    BB
# zz    cc    CC
#
# SOMA DataFrame schema:
# soma_joinid: int64 not null
# idx: large_string
# col0: large_string
# obs_id: large_string
# metadata[soma_dataframe_original_index_name]:
# "idx"
#
# After:
#     col0 obs_id
# idx
# xx    aa     AA
# yy    bb     BB
# zz    cc     CC

DataFrame also has a column named id_column_name (default: obs_id)

6. df.index is unnamed:

unnamed index → column named level_0
obs_id column → unnamed index
"index" column dropped

python obs-roundtrip.py -i xx,yy,zz -c obs_id=aa,bb,cc -c index=AA,BB,CC 2>/dev/null
# Before:
#    obs_id index
# xx     aa    AA
# yy     bb    BB
# zz     cc    CC
#
# SOMA DataFrame schema:
# soma_joinid: int64 not null
# level_0: large_string
# obs_id: large_string
# metadata[soma_dataframe_original_index_name]:
# null
#
# After:
#    level_0
# aa      xx
# bb      yy
# cc      zz

7. df.index has a name:

"index" column is dropped

python obs-roundtrip.py -i idx=xx,yy,zz -c obs_id=aa,bb,cc -c index=AA,BB,CC 2>/dev/null
# Before:
#     obs_id index
# idx
# xx      aa    AA
# yy      bb    BB
# zz      cc    CC
#
# SOMA DataFrame schema:
# soma_joinid: int64 not null
# idx: large_string
# obs_id: large_string
# metadata[soma_dataframe_original_index_name]:
# "idx"
#
# After:
#     obs_id
# idx
# xx      aa
# yy      bb
# zz      cc

It's not clear what backwards-compatibility concerns might exist from fixing each case.

It's not clear what backwards-compatibility concerns might exist from fixing each case.

Our existing unit-test suite is our bulwark against all bugs in this area that have been reported and fixed. As long as we can add new unit-test cases for new fixes, without any old unit-test cases breaking, we should be good.

None of the "examples" above have been addressed. They apply to DataFrames in obs, var, and uns, but only when certain column-/index-name conditions are met.
#2874 fixed two separate issues that were specific to uns DataFrames:

Users' uns DataFrames were mutated on ingest (soma_joinid index added)
uns DataFrames were modified by I/O round trips (soma_joinid column added)

These applied to all uns DataFrames, regardless of index-/column-names. There was never an issue filed about them, just some Slack discussion.

Thanks @ryan-williams !


Removed CLI component, updated docs, tests, removed abandoned dependencies

Removed CLI component, updated docs, updated tests, removed abandoned dependencies.
I also removed the composer.lock, as this is not used for packages but only for implementing projects.
Fixes #6

Thanks! :)


Removing ufunc-broadcasting across record fields

Currently, all ufuncs are broadcasted across all fields of a record:
>>> ak_array = ak.Array([{"x": 1, "y": 1.1}, {"x": 2, "y": 2.2}, {"x": 3, "y": 3.3}])
>>> ak_array
<Array [{x: 1, y: 1.1}, ... {x: 3, y: 3.3}] type='3 * {"x": int64, "y": float64}'>

>>> ak_array + 1
<Array [{x: 2, y: 2.1}, ... {x: 4, y: 4.3}] type='3 * {"x": int64, "y": float64}'>

This is causing some confusion because the fields of a record have qualitatively different meanings. Some are trigger booleans, some are momenta, some are ML-derived isolation variables, some are strings...
>>> ak.Array(["HAL"]) + 1                      # should this even work?
<Array [[73, 66, 77]] type='1 * var * uint8'>

>>> [chr(x) for x in (ak.Array(["HAL"]) + 1)[0].tolist()]
['I', 'B', 'M']

Furthermore, when @henryiii is writing vector, he has to distinguish between LorentzVector + LorentzVector accidentally working because they're Cartesian (but not preserving their Lorentzness) and getting the wrong answer because they're not Cartesian. Even though the + behavior is defined, due to the fact that they are records, he has to be sure to override every case.
I think there would be fewer surprises for both users and developers if broadcasting a ufunc through a record were an error (withe a nice error message). Custom behaviors for specialized records, like LorentzVectors, would still be possible to define, as they are now, but instead of replacing wrong behavior, they'd be replacing no behavior.
Note that NumPy does not define such an operation on structured arrays:
>>> np_array = np.array([(1, 1.1), (2, 2.2), (3, 3.3)], [("x", int), ("y", float)])
>>> np_array
array([(1, 1.1), (2, 2.2), (3, 3.3)], dtype=[('x', '<i8'), ('y', '<f8')])

>>> np_array + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: invalid type promotion

Although it does work for Pandas:
>>> df = pd.DataFrame({"x": [1, 2, 3], "y": [1.1, 2.2, 3.3]})
>>> df
   x    y
0  1  1.1
1  2  2.2
2  3  3.3
>>> df + 1
   x    y
0  2  2.1
1  3  3.2
2  4  4.3

it is not our intention to generalize from Pandas, only NumPy.
This would also affect ufuncs that return booleans, like comparison operators. For these, the argument isn't as strong. Maybe we want
>>> ak_array > 1
<Array [{x: False, y: True}, ... y: True}] type='3 * {"x": bool, "y": bool}'>

to work, maybe we don't.
I'm considering removing ufuncs-through-records for all ufuncs, without affecting the custom ufunc behavior that can be assigned to any record with a name. (I'm not considering ufuncs-on-strings right now, though that's something to think about.) Does anyone have a strong argument about that?
(I suppose this needs a deprecation cycle, though it would be a little difficult getting a warning into the middle of the broadcast-and-apply.

For the record I did propose also to only disable ufuncs on arrays where layout.purelist_parameter("__record__") == None but I learned that this name is used for other purposes than just defining the desired behavioral mixin (e.g. if multiple differently-named record is encountered in ArrayBuilder, then the resulting array will be a union of each record type rather than a single record type with optional subfields).

Also, I want to keep these rules of "what applies when" as simple as possible, to minimize surprising behavior.

I'm not going to change the policy on this: the behavior will stay the same. (Although NumPy ufuncs won't be allowed on strings, as in the above example: #504.)

@lgray FYI
Isn't the risk of user error extremely large here?

So yeah, my 2 cents here is that the probability of the user doing something they don't intend is incredibly large.
Nick ran into an interesting case where NanoEvents jets (which are four vectors beneath) can be element-wise multiplied by each other. This doesn't have a defined behavior from a physics standpoint and would also be very easy to accidentally do or do intentionally without knowing what it meant. In either case it can result in technically valid but wrong behavior when writing analysis - which is a much less preferred outcome than throwing an error.

The way it works now is more prone to error for the interpretation of records as objects. "Extremely" is subjective.
Heck, in crafting any kind of response to this, I'm only convincing myself that it needs to be done. It is the case that examples of this antifeature are actually in some of my slides, so "fixing" it will make my slides wrong.
But it is an important point. I think this issue could be implemented as an inhibitor that prevents broadcasting through any records unless a customization is defined. Note that that even means this:
>>> array = ak.Array([{"only": 1}, {"only": 2}, {"only": 3}])
>>> array + 10
<Array [{only: 11}, {only: 12}, {only: 13}] type='3 * {"only": int64}'>

If/when people complain that what used to work no longer works, I'll just have to explain that. (In the above example, they'd have to do array.only + 10.)

I'm calling it a bug because I've just defined the current behavior as wrong. (Even though I've presented it in talks.)
(A motivator for being short with these things is that the list of issues is a lot longer than I thought, and I have only until December 1 to make this awkward==1.0.0.)

Yeah - I agree that a good policy is to not allow broadcasting ufuncs through records unless that ufunc has been defined for that record type. This has the added benefit that people have to agree on the ufuncs/interface for whatever domain-specific set of records they want to use.

@henryiii I'm working on this now: broadcasting will be prevented from passing through records unless an overload has been implemented. For example, array_of_records * 10 won't multiply all fields of the records by 10 unless those records have explicitly defined a behavior for np.multiply. Thus, a Lorentz vector object in cylindrical won't accidentally get its phi member multiplied by 10, but the cartesian object would have to explicitly define this.
It is a change in interface, and I've even presented examples of this interface in public, so I have to Do The Right Thing and give it a deprecation cycle. It's removing a feature, which is good because changing a feature doesn't have a way to provide both old and new behaviors for an overlapping time period. What will happen is this: if any function attempts to broadcast through a record, it will raise a warning now, saying that this will become an error in version 1.0.0 on Dec 1, 2020, unless that broadcast is an overloaded ufunc.
Then I'm going to push this change into a released version of Awkward Array right away, so that about a month of releases will get the message. I'm sure there are users who update less frequently than that, and unfortunately, they won't get the message. Post-1.0.0, I'll have to start publishing regular release schedules, so that deprecation warnings can refer to both a version number and a date.

In Awkward Array 0.4.1, which I want to release later today, the warning that you get from
one = awkward1.Array([{"x": 1}, {"x": 2}, {"x": 3}])
two = awkward1.Array([{"x": 1.1}, {"x": 2.2}, {"x": 3.3}])
one + two

is
DeprecationWarning: In version 1.0.0 (target date: 2020-12-01), this will be an error.
(Set ak.deprecations_as_errors = True to get a stack trace now.)

ValueError: cannot broadcast: <class 'awkward1._ext.RecordArray'>, <class 'awkward1._ext.RecordArray'>

(https://github.com/scikit-hep/awkward-1.0/blob/0.4.1/src/awkward1/_util.py#L886)

But
def overload_add(left, right):
    return awkward1.Array({"x": left.x + right.x})

behavior = {}
behavior[numpy.add, "Overload", "Overload"] = overload_add

one = awkward1.Array([{"x": 1}, {"x": 2}, {"x": 3}], with_name="Overload", behavior=behavior)
two = awkward1.Array([{"x": 1.1}, {"x": 2.2}, {"x": 3.3}], with_name="Overload", behavior=behavior)

assert (one + two).tolist() == [{"x": 2.1}, {"x": 4.2}, {"x": 6.3}]

raises no warnings or errors.

Looks like the correct behavior to me. :-)


Adding test case for aggregate exception

This PR is a remake of #142 to pull to master instead of the PassExceptionsOnTaskFaults branch.

Looking now. Thanks!

After pulling in your changes and the changes to TestAdapter by @drub0y , my test required no changes. Let me know if you notice anything off!

Thanks Ryan! I think this is a great addition to the tests.
Already tonight, with the Exception improvements to the Test Adapter, I've caught bugs that previously were slipping through. Having the exceptions flow through correctly has been a huge help!

@cleemullins  Sweet. I'll try and monitor the issues if you post them.


Update README.md

Minor edits and fixes.

Thanks for updating the file.

Thanks for updating the file.

Any possible bounties for some Onions?


Publish to crates-io?

This crate is awesome! Would you like to publish it so that crates depend on this one could be published?

@cryscan
I am really really sorry it took so long to respond! Also thank you for the kind words. I had to hopes to clean up the project before publishing but I just never actually had the time so I guess I will just publish it in the current state.
https://crates.io/crates/repugnant-pickle
Please let me know if you run into issues using that and hopefully it won't take so long to get a reply.


usb.core.USBError: [Errno 32] Pipe error

I am trying to read/write using a USB-OTG port in Altera De0 Nano SoC. I have the Atlas SoC installed in a SD card for using the Arm Linux.
 The drivers removed by me are: cdc_acm, rndis_wlan, rndis_host. I am using the device as USB Mass Storage for Bulk Transfer (Interface 4)
I am running the following python code:
#For USB OTG Hexadecimal VendorID=0x1d6b & ProductID=0x104
dev = usb.core.find(idVendor=0x1d6b, idProduct=0x104)
interface=4
if dev is None:
    sys.exit("No Panic button found in the system");

try:
    if dev.is_kernel_driver_active(4) is True:
        dev.detach_kernel_driver(4)
except usb.core.USBError as e:
    sys.exit("Kernel driver won't give up control over device: %s" % str(e))

try:
    dev.set_configuration()
    dev.reset()
except usb.core.USBError as e:
    sys.exit("Cannot set configuration the device: %s" % str(e))
print dev
endpoint_in = dev[0][(interface,0)][0]
endpoint_out = dev[0][(interface,0)][1]
print endpoint_in
print endpoint_out
msg = 'test123'
data1 = dev.write(endpoint_out, msg,100)
print data1
data = dev.read(endpoint_in, len(msg),100)
sret = ''.join([chr(x) for x in data])
print sret
print repr(data.tostring())
usb.util.dispose_resources(dev)

I am able to write to the device using "data1 = dev.write(endpoint_out, msg,100)" but when I try reading from the device "data = dev.read(endpoint_in, len(msg),100)", I am getting the following error:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/usb/core.py", line 988, in read
self.__get_timeout(timeout))
File "/usr/local/lib/python2.7/dist-packages/usb/backend/libusb1.py", line 833, in bulk_read
timeout)
File "/usr/local/lib/python2.7/dist-packages/usb/backend/libusb1.py", line 936, in __read
_check(retval)
File "/usr/local/lib/python2.7/dist-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 32] Pipe error
I tried to search for this issue but could not find much.
In http://christopherpeplin.com/2012/02/bulk-usb-throughput, the following is mentioned:

The complete fix for OpenXC’s use case will be to make sure every message is padded out to 64 bytes and to request bulk transfer sizes that are big enough to get good throughput but small enough to not be too delayed. I have yet to confirm, but my understanding is that if we request a 4KB read and don’t mark the end of the transfer with a less-than-max length packet, the host device will block waiting for the rest of the requested data.

I tried to send a 64 byte data using the following string:
abcdabcdabcdabcdabcdabcdabcdabcdabcdabcdabcdabcdabcdabcdabcdabcd
The write command shows 64 bytes written but the same error 32 arises when I try to read from the device. I am new to USB programming and I am stuck up here. Can somebody please help?

You need to understand the communication protocol in order to use libusb/pyusb. USB mass storage device is not a good start to use libusb/pyusb. You can read the libsb faq here.
https://github.com/libusb/libusb/wiki/FAQ#Can_I_use_libusb_to_open_a_file_on_an_USB_storage_device

What is your objective? If your objective is to use pyusb and libusb, then this is not a good device to start with. pyusb/libusb are more suitable for generic device not fall into specific class (eg: RNDIS, CDC-ACM and USB Mass Storage).

As long as you know the communication protocol or you have control on your device side (it seems to me that you have control since this is a development board), then you can use the interfaces to communicate with the device, for example, CDC-ACM will be the serial port and you can use pySerial for the communication. rndis will come out as the network interface and you can use those network related python (or other language) packages for the communication.

@mcuee Thanks a lot for your guidance - I got the thing working using pyserial

Glad that you solved the problem and thanks for reporting back.


FR: Add avantgardefuselage repo

Someone linked to this one in Slack:
https://github.com/avantgardefuselage/macos-profiles
Looks pretty new!

Thanks! 🎉


[DOCS] Update documentation step 13.

Description
##current doc

Click 'compare across forks' and you will see 4 drop downs
but github doesn't do this anymore

Screenshots

Additional information
No response

Hi @lilmubby , I would like to work on this. Would you mind suggesting the correct approach for this, I was thinking about changing it so that it says to choose a base repo and a base branch along with a screenshot displaying the same , like this

I can solve it can anyone assign it to me?

hi @Avi-88 please you can go ahead

@khalidx3 please have a look at the documentations, there's a good issue there

Hi @lilmubby  sorry for the delayed reply, I think we can just replace the current representation of the dropdowns with the actual screenshot. Please let me know what you think of this


The Hangman Game added

closes #16
Changes Proposed
Added project to project folder
Updated Index.html with project

Delete vs code live server file

fixed and applied chnages

Delete vs code live server file

@TusharKesarwani  fixed and applied chnages


Add SwiftLogNoOpLogHandler.init(_: String)

Motivation:
LogHandler facotories are given a label as input and return a
LogHandler. Since the no-op handler doesn't care about the label
callers must provide a thunk: { _ in SwiftLogNoOpLogHandler() }.
That's fine but slightly more annoying to type than it needs to be.
Modifications:

Add SwiftLogNoOpLogHandler.init(_: String)

Result:
Users can create a logger with the no-op handler with:
Logger(label: "foo", factory: SwiftLogNoOpLogHandler.init)

instead of:
Logger(label: "foo", factory: { _ in SwiftLogNoOpLogHandler() })

Do people mind doing this on in 1.4.3 or should we 1.5.0? Technically it's new API

Do people mind doing this on in 1.4.3 or should we 1.5.0? Technically it's new API

Let's do 1.5.0, we don't need to release 1.4.3 at all.

+1 for 1.5.0, version numbers are cheap!

Cool, I can cut that tomorrow morning, unless you're in a rush :-0

No rush at all, this was just an itch I needed to scratch.


Fix download file issue

@RocketChat/core

Closes #382
Closes #388

Wasn't detecting relative urls as internal.

Hi Guys,
I'm trying to locate the preload.js file.
Could you explain me where I can find it?
I install the server on a ubuntu 16.04LTS with snap.
Is-it a file of the server or of the client?
Thanks in advance.
Alex

This isn't really the place for this question.  But the file is in the desktop client: https://github.com/RocketChat/Rocket.Chat.Electron/blob/develop/src/public/preload.js  Please open an issue for follow up questions.

Thank you sorry for this.


Propose a tenant

Codecov Report

Merging #218 (9d17f1b) into main (bcc2fa5) will decrease coverage by 0.60%.
The diff coverage is 4.87%.

@@            Coverage Diff             @@
##             main     #218      +/-   ##
==========================================
- Coverage   81.13%   80.53%   -0.61%
==========================================
  Files          58       58
  Lines        5233     5272      +39
==========================================
  Hits         4246     4246
- Misses        987     1026      +39

Impacted Files
Coverage Δ

pallets/asset_management/src/lib.rs
53.62% <0.00%> (-21.13%)
:arrow_down:

pallets/roles/src/tests.rs
100.00% <100.00%> (ø)

pallets/roles/src/types.rs
94.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


collectionOperations "get" not declared by design of backend

API Platform version(s) affected: 0.3.0
Description

px @api-platform/client-generator -g vuetify https://backend.localhost/api src/
Error: Unable to find the URL for "https://backend.localhost/api/docs.jsonld#Entrypoint/cloudProvider", make sure your api resource has at least one GET item operation declared.
    at _loop_1 (/Users/alumni/.npm/_npx/51964/lib/node_modules/@api-platform/client-generator/node_modules/@api-platform/api-doc-parser/lib/hydra/parseHydraDocumentation.js:212:23)
    at /Users/alumni/.npm/_npx/51964/lib/node_modules/@api-platform/client-generator/node_modules/@api-platform/api-doc-parser/lib/hydra/parseHydraDocumentation.js:232:13
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

file admin/src/Entity/CloudProvider.php:
/**
 * @ApiResource(
 *     attributes={"security" = "is_granted('ROLE_USER')"},
 *     collectionOperations={
 *          "post"
 *     },
 *     itemOperations={
 *          "get" = {
 *              "security" = "is_granted('VIEW', object)",
 *              "security_message" = "Only the owner can view this object"
 *          },
 *          "put"={
 *              "security" = "is_granted('EDIT', object)",
 *              "security_message" = "Only the owner can edit this object"
 *          },
 *          "delete" = {
 *              "security" = "is_granted('DELETE', object)",
 *              "security_message" = "Only the owner can delete this object"
 *          }
 *     }
 * )
 * @ORM\Entity(repositoryClass=CloudProviderRepository::class)
 * @ORM\EntityListeners({"App\Doctrine\SetOwnerListener"})
 */

Possible Solution

I believe that we should just ignore not declared GET api resource because design frontend.
Workaround:
/**
 * @ApiResource(
 *     attributes={"security" = "is_granted('ROLE_USER')"},
 *     collectionOperations={
 *          "get" = {
 *              "description" = "This endpoint exists only because of api-platform/client-generator",
 *              "security" = "is_granted('ROLE_ADMIN')",
 *              "security_message" = "This endpoint is only meant to be accessed by admins"
 *          },
 *          "post"
 *     },
 *     itemOperations={
 *          "get" = {
 *              "security" = "is_granted('VIEW', object)",
 *              "security_message" = "Only the owner can view this object"
 *          },
 *          "put"={
 *              "security" = "is_granted('EDIT', object)",
 *              "security_message" = "Only the owner can edit this object"
 *          },
 *          "delete" = {
 *              "security" = "is_granted('DELETE', object)",
 *              "security_message" = "Only the owner can delete this object"
 *          }
 *     }
 * )
 * @ORM\Entity(repositoryClass=CloudProviderRepository::class)
 * @ORM\EntityListeners({"App\Doctrine\SetOwnerListener"})
 */

Additional Context

Related bug issue in another project https://github.com/Sylius/Sylius/issues/11562

This issue is not related to the client generator but by the api design.
To be able to access a collection of a ressource you should have at least a GET item operation to make it working well (due to the IRI concept). If you don't need the GET operation for your collection, instead of omitting it altogether, you should instead declare a GET item operation which returns HTTP 404 (Not Found).
For more information, take a look at the documentation.


send a value to the bt-Device

Hi,
i am using your example for a Radon sensor (Radoneye). So far i can read the last value from the sensor with this. But my sensor needs to write a value to a characteristic in order to send a produce an updated value. Any idea how i could manage this with BLETracker?
The code would be:
BLERemoteCharacteristic* p2RemoteCharacteristic;
...
p2RemoteCharacteristic = pRemoteService->getCharacteristic(char24UUID);
if(p2RemoteCharacteristic->canWrite()) {
p2RemoteCharacteristic->writeValue(0x50);
After this a new sensor value is send on a second characteristic.
Thaks for your help,
Michi

Sorry, I don't have an idea how to do this. I guess this functionality will be added sometime as a native component, and it will probably already be possible now with the right lambda. Maybe try asking on the ESPHome forum?

Coincidentally, I just found this: https://github.com/esphome/esphome/pull/2200#issuecomment-962559276
This looks like you could use it for your purpose too.

Thanks for your help, in between somebody wrote an integration for my sensor direct.
https://esphome.io/components/sensor/radon_eye_ble.html


Memory allocation/free issues ?

Reported by karl smith on 19 Jul 2017 01:40 UTC
Using the new memory functions and MEMORYDEBUG compiler flag, I was able to monitor the allocation and free-ing of memory in ferret code.  (This prints out addresses allocated and freed by these functions.)    Not a big deal for Ferret, and may be okay since I didn't explicitly cancel everything before quitting.
However, there are a couple of pointers that are freed but never allocated.  Since this did not cause a crash, I suspect it is memory allocated from elsewhere (netcdf library?), but need to verify.  And if so, may need to use C's free for that instead of Python's PyMem_Free when in PyFerret.
Since the 32-bit version is showing a crash on memory allocation(!) of a small amount of memory in the run_tests testing, there is some memory corruption going on somehow - at least in the 32-bit build.  Weird that a malloc crashes.  Assuming something wrote past allocated memory earlier and messed up the allocation table.  Using valgrind may help figure out this problem once the specific triggering script (in bn720_bug_fixes.jnl) is found.
FYI - the file output from defining MEMORYDEBUG can be split into two - allocated and freed - and each sorted, then diff or meld to see what does not match.
awk '/alloc/ {print $1}' memorydebug.txt | sort > alloc_addresses.txt
awk '/free/ {print $1}' memorydebug.txt | sort > free_addresses.txt

Intending to work further on this when I can, but leaving it unassigned for now.  Set to high priority and major severity just for checking.  No crashes observed with 64-bit version so might all be okay.
Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/2555

Comment by karl.smith on 19 Jul 2017 02:26 UTC
Note that memorydebug.txt is always appended to.  So need to delete it between tests.
Using ferret -script test.jnl to run tests, a session where I just opening the dataset and then canceling it show only a couple of pointers not freed, and no free of unallocated pointers.  Adding the shade command added a lot more pointers allocated but not freed, even with a 'can data /alll', 'can var /all', and 'can mem /all' added to the end, but still no unallocated pointers show up.  Similar with 'go fland' added.
However, starting ferret, then at the ferret prompt running 'go "./test.jnl"', and then quitting, shows a "changed" pointer (pointer allocated but not freed and pointer freed but not allocated that happens to be in the same position in the sorted list).
With -script (so 'is_server') the readline library is skipped and memory for the line read is allocated and later freed by Ferret.  With interactive, readline is handing back input lines in memory it has allocated which is later freed by Ferret.  So this is certainly the source of the unallocated addresses.  Will modify code to remove this 'red herring'.

Comment by karl.smith on 21 Jul 2017 04:27 UTC
Added calling source file names and line numbers to the print-out to make it more useful.  But need to tweak the awk scripts for the filenames:
awk '/ allocated / {print $1}' memorydebug.txt | sort > alloc_addresses.txt
awk '/ freed / {print $1}' memorydebug.txt | sort > free_addresses.txt

Ansley found the following crashes ferret on a malloc
use coads_climatology
let allatts = sst.attnames
let natts= sst.nattrs
 REPEAT/RANGE=1:`natts`/NAME=m (LET attname = allatts[list attname)

But that can be trimmed down a lot and still give the crash
use coads_climatology
list sst.attnames[I=1](I=`m`];)

Comment by karl.smith on 21 Jul 2017 05:04 UTC
And PyFerret comes up with lots "String array not initialized yet!" messages, which I now have set to abort in debug mode.
Just starting up and then shutting down cause a lot of memory allocations from string_array_init and deleted_list_... (now that my messages include filename and line number).  Need to investigate if this is correct.

Comment by karl.smith on 21 Jul 2017 18:25 UTC
Ends up  the Ferret and PyFerret problems are the same bug.  Steve used INTEGER8 for the size of memory requests.  Which actually does correspond to size_t == long.  But C routines have been calling the corresponding reference argument int instead of long*.  So PyFerret was reading zero and Ferret was getting something else but not the correct thing.  I had caught one of these some time ago but had forgotten about checking that again.
So going through to make sure everything INTEGER8 corresponds to a long.  And probably need to make sure the same in Fortran.  All the Fortran INTEGER*8 function arguments now start with i8_ to make this clear.

Comment by karl.smith on 22 Jul 2017 03:11 UTC
Well, that needed fixing but is not the cause of the problem after all.  Testing with the simple ```list sst.attnames[script last mentioned.  But something going wrong with init_c_string_array_ causing the issue in both programs.
Using script  test2.jnl
use coads_climatology
list sst.attnames[I=1](I=1]```)
can data /all
can var /all
can mem /all
exit

and the command
valgrind --read-var-info=yes --leak-check=full --show-leak-kinds=all `which ferret` -script test2.jnl > & valgrid.log

shows the summary
==64544== Invalid write of size 8
==64544==    at 0xDC796C: init_c_string_array_ (init_c_string_array.c:55)
==64544==    by 0x9D88F2: is_attrib_val_ (is_attrib_val.F:349)
==64544==    by 0x9C24E0: interp_stack_ (interp_stack.F:548)
==64544==    by 0x9BC10E: get_prot_cmnd_data_ (get_prot_cmnd_data.F:215)
==64544==    by 0x7E7441: xeq_list_ (xeq_list.F:326)
==64544==    by 0x467614: ferret_dispatch_ (ferret_dispatch.F:208)
==64544==    by 0x93EDB6: ferret_dispatch_c (ferret_dispatch_c.c:78)
==64544==    by 0x466A50: command_line_run (fermain_c.c:489)
==64544==    by 0x4666B7: main (fermain_c.c:402)
==64544==  Address 0x117c6b88 is 0 bytes after a block of size 8 alloc'd
==64544==    at 0xADDCBE3: malloc (vg_replace_malloc.c:299)
==64544==    by 0x46859F: FerMem_Malloc (FerMem_routines.c:45)
==64544==    by 0xDC7343: get_mr_mem_ (get_mr_mem.c:24)
==64544==    by 0xD23EDB: get_mr_dynmem_ (get_mr_dynmem.F:83)
==64544==    by 0xD1E12B: create_temp_mem_var_ (create_temp_mem_var.F:86)
==64544==    by 0x9D5A38: is_attrib_val_ (is_attrib_val.F:112)
==64544==    by 0x9C24E0: interp_stack_ (interp_stack.F:548)
==64544==    by 0x9BC10E: get_prot_cmnd_data_ (get_prot_cmnd_data.F:215)
==64544==    by 0x7E7441: xeq_list_ (xeq_list.F:326)
==64544==    by 0x467614: ferret_dispatch_ (ferret_dispatch.F:208)
==64544==    by 0x93EDB6: ferret_dispatch_c (ferret_dispatch_c.c:78)
==64544==    by 0x466A50: command_line_run (fermain_c.c:489)
==64544==
 PERMANENT data cleared from memory
 TEMPORARY data cleared from memory
             VARIABLE : SST.ATTNAMES[I=1]
             FILENAME : coads_climatology.cdf
             FILEPATH : /home/merlot/ksmith/Datasets/data/
             X        : 1
        "missing_value"
==64544==
==64544== HEAP SUMMARY:
==64544==     in use at exit: 0 bytes in 0 blocks
==64544==   total heap usage: 23,607 allocs, 23,607 frees, 16,372,111 bytes allocated
==64544==
==64544== All heap blocks were freed -- no leaks are possible
==64544==
==64544== For counts of detected and suppressed errors, rerun with: -v
==64544== ERROR SUMMARY: 4 errors from 1 contexts (suppressed: 0 from 0

At least I got all the leaks addressed for this script.
If Xgks starts up there are leaks there but I don't care since it goes away in PyFerret.
Using PyFerret is deceptive since Python memory management the uses garbage collection model.
So the PyMem_Free means Python is free to do what it wants with the memory and does not mean it is given back to the OS like free in C.

Comment by karl.smith on 23 Jul 2017 00:04 UTC
Finally figured out the cause of this bug.  Not surprising, memory is being written to past its allocated amount.  Not sure of the best way to fix it.
The init_c_string_array assumes that the memory for its array of pointers has already been allocated. and just records the pointer to this array of pointers, then initialized the array of pointers with NULL.  The memory for this array was allocated earlier (at least in is_attrib_val.F) by a call to CREATE_TEMP_MEM_VAR.  The amount of memory allocated is computed from the cx... dimensions by R_CGRID_SIZE_DELTA.  So this is all good when the variable to be read in is appropriately described by the cx.... dimensions.
But in this error in IS_ATTRIB_VAL, the call to CREATE_TEMP_MEM_VAR is only allocating memory for one pointer (8 bytes) and the init_c_string_array call is assuming there is room for five pointers (40 bytes) and writes NULL past its memory.
From what I can tell, in this case there does need to be memory allocated for five pointers.  So something needs to be done to have enough memory allocated.  Not sure if this is modifying the cx... dimensions before calling the CREATE_TEMP_MEM_VAR, if CREATE_TEMP_MEM_VAR (also) needs to be moved later in the code, possibly duplicating for each ELSEIF, or if some other routine needs to be used to allocate the correct amount of required memory.  I am assume CREATE_TEMP_MEM_VAR still needs to be called since these attributes are memory variables, so highly unlikely that something new needs to be created.
Compile the ferret make target memorydebug and test (after deleting any memorydebug.txt file) the resulting executable with the valgrind command shown above.  The valgrind.log file will show if there is still an illegal memory write.  The memorydebug.txt file will contain a comment like the following:
...
0x000000117c1e60 : 1 : memory malloc allocated for 60 bytes : file list.c : line 186
0x000000117c7b80 : 1 : memory malloc allocated for 8 bytes : file get_mr_mem.c : line 24
init_c_string_array: pointer 0x000000117c7b80 to be assigned as an array of 5 pointers (40 bytes) initialize to NULL
0x000000117c9f90 : 1 : memory malloc allocated for 14 bytes : file save_c_string.c : line 51
0x000000117ca260 : 1 : memory malloc allocated for 8 bytes : file get_ws_mem.c : line 26
...

The size allocated for the pointer in that init_c_string_array statement should match the size mentioned in that statement.
In this case, the allocation line (just above the init_c_string statement) shows only 8 bytes allocated instead of 40.

Comment by karl.smith on 23 Jul 2017 14:35 UTC
Looking at the code for is_attrib_val.F, I am thinking (but not sure) the two sections where it is assigning a string array with dimension nvatts or nvdims are the only sections at fault.  (Guessing that nvdims will also have the same problem.)  The original CREATE_TEMP_MEM_VAR only expects one pointer but nvatts or nvdims pointers are being assigned.
Not sure if the original temp var needs to be cancelled, cx... values changed, and a new temp var created to hold these.  Or maybe this temp var should just be holding a single pointer to a new temp var of the appropriate size holding the string arrays.
Assigning this over to Ansley as I am sure she better understands this part of the code and figure out what needs to be changed.

Comment by ansley.b.manke on 26 Jul 2017 16:36 UTC
You're right. This is exactly where the bug was, in is_attrib_val.F, the sections where it makes calls like INIT_C_STRING_ARRAY(natts,...  and likewise calls using ngatts, nvdims.
The right number of dynamic memory slots have been set up, early in is_attrib_val.F, with a call to CREATE_TEMP_MEM_VAR( cx, mr, status ), where it gets the number of memory slots needed for the context, i=1 or i=2:3, whatever is requested.
But then when the calls to INIT_C_ARRAY are made, the total number of attribute or dimension -name strings was used in several places, rather than the number requested by the context, so undefined pointers were being initialized. The fix is that the calls to INIT_C_ARRAY should be done with i2-i1+1, not nvatts.

Comment by karl.smith on 31 Jul 2017 17:13 UTC
The call to INIT_C_ARRAY should use whatever number of strings are going to be assigned, which hopefully is i2-i1+1, but there are still cases where this is not true.  By using i2-i1+1 you are indeed only initializing valid memory locations, but if that is smaller than the nvars (for the list of variable names returned by ..varnames) and you are assigning nvars strings, you are still assigning values (pointers to strings) to invalid memory locations (and the problem becomes harder to detect).  I would recommend figuring out how many strings are going to be assigned and testing that is equal to i2-i1+1 before the INIT_C_ARRAY call, or test afterwards if cannot do it beforehand.
What I am seeing in PyFerret appears to be exactly that with ..varnames.

Comment by ansley.b.manke on 31 Jul 2017 17:23 UTC
huh.  That sounds like the context-setting code is incorrect, if it's setting the numbers that become i1 and i2 such that the number is larger than the number of variable names.

Comment by karl.smith on 31 Jul 2017 18:47 UTC
Sorry, but now looking the issue I am seeing on the Ferret side of things is that I am getting lots of
String array not initialized yet (string_array_find_quoted)!
When I was seeing this before, it was due to this writing in invalid memory, so checks that there was enough memory allocated would still be good to add otherwise hard to detect this sort of bug.
There is still the issue with the pyvars that I need to figure out.

Comment by karl.smith on 31 Jul 2017 19:15 UTC
Need to figure out the true cause of that error message since it the is_attrib_val.F code appears to be correct.  Back to valgrind and memorydebug.

Comment by ansley.b.manke on 31 Jul 2017 19:23 UTC
ok, I was just going to say - I Never see the "String array not initialized yet" from the Ferret trunk, but I do from PyFerret.
Let me know if I can help.

Comment by karl.smith on 9 Aug 2017 22:10 UTC
Still some memory issues in Ferret.
Added valgrind-like memory monitoring when the compiler flag MEMORYDEBUG is defined.  Running the benchmark scripts using a memorydebug build (which does run much slower) of pyferret or ferret shows writing past end of allocated memory errors, so will look into fixing those issues before getting too deep into figuring out why only pyferret is showing the "String array not initialized yet" message.  The benchmark tests giving this error:
bn_internal_external_functions.jnl
bn_delimited_read.jnl
bn_cancel_axes.jnl
bn_attributes.jnl
bn614_bug_fixes.jnl
bn64_bug_fixes.jnl
bn685_bug_fixes.jnl
bn69_bug_fixes.jnl
bn_delim_datetime.jnl
bn_ez_noupcase.jnl
bn720_bug_fixes.jnl

Comment by karl.smith on 10 Aug 2017 05:10 UTC
All of these are fixed by adding one more character to a string allocation for the null character at the end of a C string in ez_delimited_read.c, line 981.

Comment by karl.smith on 10 Aug 2017 23:54 UTC
The PyVar's (ferret variables from Python) under fer/pyf in PyFerret need to be updated to the new memory design.  Realized that this is probably where that message is coming from since it is still using the old design.  Although having nothing to do with the particular things being done, at some point the list of PyVar's will be searched and would generate such a message.  This goes along with everything looking fine (except for these messages) until tests involving PyVar's.

Comment by karl.smith on 15 Aug 2017 03:19 UTC
Re-added the initialization of the pyvar list in initialize_ferret.F, and the clearing in finalize_ferret.F.
The pyvar additions were lost in the renames from initialize.F and finalize.F; I missed the addition I had made previously.
Also changed to do a proper clean-up on exit directly from the ferret prompt.
This has fixed all the pyferret-specific issues.
More info added to memory not freed, but the error is only reported and does not cause ferret/pyferret to return errors.  Thus, with MEMORYDEBUG, the benchmark tests run successfully although they include error message about memory allocated but not freed.  Of no consequence for Ferret, but a memory leak for PyFerret as a python module.
Going to close this out and generate a new issue in GitHub specific to this last issue.


Lab04-Task03 issue.

ラボ/デモ:04
タスク:03
The instruction needs to be changed due to an update of the Azure Portal. The English version has already been updated.
https://github.com/MicrosoftLearning/AZ-104-MicrosoftAzureAdministrator/issues/713
https://github.com/MicrosoftLearning/AZ-104-MicrosoftAzureAdministrator/commit/bcca0293bd3fa3044e8bcaf867371ac684e51982
The old instruction always causes errors and needs to be updated as soon as possible.

Hello contributor,
Thank you for your contribution! We are processing your suggestions and we will update you shortly.
Kind regards,
Microsoft Worldwide Learning Team

WWL PM Follow-up needed - these locfiles will need to be handed off so the updates can be viable for the loc langs
Query for locfiles for ja-jp

we are now continuously publishing this repo, so this issue should now be fixed, along with English content.


Upgrade PostgreSQL instances to 9.5.13

Since PostgreSQL versions 10.4, 9.6.9, 9.5.13, and 9.4.18 contain important cumulative security fixes, we recommend that you upgrade your PostgreSQL 10.x instances to 10.4, PostgreSQL 9.6.x instances to PostgreSQL 9.6.9, PostgreSQL 9.5.x instances to PostgreSQL 9.5.13, and PostgreSQL 9.4.x instances to PostgreSQL 9.4.18 or higher at your earliest convenience. If you are running PostgreSQL 9.3.x, please note that support has been discontinued and we recommend to upgrade to 9.4.x or above as early as possible.

[x] Staging
[x] Production

Looks like this already got handled. The console reports that the instances are using engine 9.5.13, and I confirmed that the Terraform remote state reflects this change for both staging and production.


kubeflow pipelines should support dex LDAP authentication

After PR https://github.com/elyra-ai/elyra/pull/2167, we no longer support Kubeflow Pipelines clusters using LDAP (only staticPasswords).
Since we are planning to have a proper authentication type dropdown (see issue: https://github.com/elyra-ai/elyra/issues/2107) in the future, this will be less of a problem, as users can specify what type of auth their cluster uses.
However, so that elyra 3.2.0 doesn't break Kubeflow Pipelines clusters (with dex LDAP authentication), we should:

Update this code to look for /auth/ldap, and proceed without forcing /auth/local

(this will make it work for the case when dex is only configured with LDAP, but will default to staticPasswords when both are configured)

I have raised PR https://github.com/elyra-ai/elyra/pull/2212 to resolve this issue.


Cannot Use

Sorry This might be silly, but I simply cannot use this plugin after I installed it with packer.nvim. When I do nvim leetcode.nvim, I just got an new file named leetcode.nvim and start to edit it with nvim. I have checked the installation by packer status and it is successfully installed.

leetcode.nvim has to pass some conditions before successfully initializing which are
Neovim argumens count must be the length of 1
That argument has to be equal to arg specified in the configuration or by default to leetcode.nvim
And lastly the buffer opened using that arg must be empty of any text
If that doesn't solve it, can you post your leetcode.nvim packer configuration?


Traking not working for Shikimori and AniList

Steps to reproduce
Just not working always. Without or with VPN.
Expected behavior
Should open login dialog
Actual behavior
Dialog in app with Error: "No Activity found to Intend { act=android.intent.action.VIEW dat=https://shikimori.one/.. pkg=com.hihonor.android.internal.app}
Crash logs
No response
Mihon version
0.16.5-6951 (standard, f3a2f566c, 7, 2024-10-19T21:19:51Z)
Android version
Android 13 (SDK 33; build ELN-L09 <IP_ADDRESS>(C185E9R2P1))
Device
HONOR Pad X9 (HNELN-Q (ELN-L09))
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have gone through the FAQ and troubleshooting guide.
[X] I have updated the app to version 0.16.5.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

Cannot reproduce, I just tested this with the same recent beta you said you were using and both my AL and my Shikimori accounts linked up successfully.

Wait a minute. You're saying you get this error when you tap the tracker name to open it? Do you have a browser on your phone?

I made an account, logged in fine, added Shikimori perfectly fine. Then I remember to update the app, but logged out & logged back in with no issues, along with adding the tracker to a manga. Same goes for AniList
Sounds like there's an issue with your own internet or device. Screen record the issue if you can, otherwise, switch networks or use a better VPN if your country blocks the tracking sites.

Did you debloat your device?

Enable/reinstall com.hihonor.android.internal.app, see: https://xdaforums.com/t/debloating-the-honor-magic-v2-global-rom.4655079/post-89326143

The package name needs to be added to https://github.com/mihonapp/mihon/blob/8ea95cb27fa3c263cc9905c63cd8493ffb831ef5/core/common/src/main/kotlin/eu/kanade/tachiyomi/util/system/DeviceUtil.kt#L65


[3pt] available options

The following options should be available:

List all movies
List all sources (e.g. 'From a friend', 'Online shop')
Add a movie

Closed this issue without implementing it as it is not required.


As a Developer  , I want to have an endpoint to generate transcripts for podcast episodes, So that I can manually trigger transcription for episodes where automatic transcription has not occurred or failed.

Acceptance Criteria:

[x] Endpoint /podcast/{episodeId}/generateTranscript is implemented.
[x] The endpoint checks if a transcript already exists for the given episode and it's not empty or corrupt.
[x] If a transcript exists, return a message indicating that it already exists.
[x] If no transcript exists, trigger the transcription process.
[x] The transcription process involves running app.add_routes([web.get('/{podcast_id}/{episode_file_name}/create_transcript', handle_transcription_request)]) from the Python container microservice.
[x] Show progress of the transcription process.

Demo Steps:

Access the endpoint with an episode ID.
Verify that it checks for an existing transcript.
If no transcript exists, observe the process of transcription initiation.
Check for the progress updates on the transcription process.
Verify the endpoint's response once the transcript is available.

Potential Risks:

[x] Endpoint misuse or overuse, leading to system overload.
[x] Transcription errors or delays in processing.
[x] Potential failure in checking the existence of a transcript.

Technical Notes:

[x] Ensure secure handling and storage of podcast episodes and transcripts.
[x] Implement appropriate authentication and authorization checks for the endpoint.
[x] Consider rate limiting to prevent abuse of the transcription service.
[x] Integrate logging and monitoring for troubleshooting and performance analysis.

Risks

Any error thrown by the transcription service are logged into a status file, which can be accessed by the ASP.net server.
If a transcript already exists for the same episode, the transcription service will not generate a new transcription.


bug with  call Redis inside queued  job

Laravel Version: 5.7.25
PHP Version:7.2
Database Driver & Version:

Description:
if i call a laravel command that connect to redis server for operation using $redis->rawCommand  every thing is working great , but once i move the same code inside a job that i queued   it will not work and get the following errror
[2019-02-21 14:06:40] local.ERROR: Class 'Redis' not found {"exception":"[object] (Symfony\\Component\\Debug\\Exception\\FatalThrowableError(code: 0): Class 'Redis' not found
it will work only if i change driver for the queue to sync ??
any help?

I'll need more info and/or code to debug this further. Please post relevant code like models, jobs, commands, notifications, events, listeners, controller methods, routes, etc. You may use https://paste.laravel.io to post larger snippets or just reply with shorter code snippets. Thanks!

Job
<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use \Redis;

class ProDevice implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    private $dtf;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(dtf)
    {
        $this->dtf = $dtf;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        ini_set('memory_limit', '-1');
       $redis = new Redis();
        $redis->connect('<IP_ADDRESS>', 8888);
        $data = $redis->rawCommand("INSPECT", "AREA", $this->dtf);
    }
}

command inside function handle()
   ` \App\Jobs\ProDevice::dispatch($dtf);`

if i run the following in  command handle , it work  ,or if job executed as sync
        ini_set('memory_limit', '-1');
       $redis = new Redis();
        $redis->connect('<IP_ADDRESS>', 8888);
        $data = $redis->rawCommand("INSPECT", "AREA", $this->dtf);

if i call a laravel command that connect to redis server for operation using $redis->rawCommand every thing is working great , but once i move the same code inside a job that i queued it will not work and get the following errror

Is it possible that you have different PHP environments (web vs. cli) and the latter is missing the redis extension?

@mfn both from cli , i don't have any web deploy i use it as cli only , laravel command is cli, and queue too

@AymanElarian Are both commands being executed on the same machine?

@ybaruchel  yes  , even same job inside queued will work with sync , will not work with database and give me the error ,
only change .env config

Does composer require ext-redis:"*" work? Can you run that on every installation where you're using the app?

@driesvints  yes its run and also its the same installation

At this point I'd recommend uploading a test app to a Github repo which re-creates the problem for you and posting the difference in settings for the different environment.


Inference

Greetings Mr @VSainteuf
During the inference, I've been following the instructions in order to do panoptic segmentation.
However, after I wrote the instruction (python test_panoptic.py --dataset_folder data --weight_folder weights/UTAE_PAPs --res_dir Output) It prompted this:
Traceback (most recent call last):
File "test_panoptic.py", line 148, in
main(config)
File "test_panoptic.py", line 97, in main
dt_test = PASTIS_Dataset(**dt_args, folds=test_fold)
File "/home/nextav/LULC/U-TAE-main/src/dataset.py", line 80, in init
if "-" in mono_date
TypeError: argument of type 'NoneType' is not iterable
So, I made some modifications; More precisely I changed the If condition by: mono_date=config.mono_date; I'm sure it's not pretty right, but it works :)
After some minutes of doing inference(and the gpu memory reached 10GB), I've faced this new error, and the Output folder stills empty:
Step [100/124], Loss: 7.8338, SQ 81.0,  RQ 53.0  , PQ 43.4
Epoch time : 289.6s
Traceback (most recent call last):
File "test_panoptic.py", line 148, in
main(config)
File "test_panoptic.py", line 134, in main
save_results(fold + 1, test_metrics, tables, config)
File "/home/nextav/LULC/U-TAE-main/train_panoptic.py", line 329, in save_results
with open(
FileNotFoundError: [Errno 2] No such file or directory: 'Output/Fold_1/test_metrics.json'
Do you have any idea for where the error could be? And thank you in advance.

@NourSoltani Hello Sir, have you succeeded in solving this problem?
Thank you in Advance.


Add map previews in the remove table dialog

Changes

Adds the vizjson URL to the template of the map card items in the delete dialog
Uses the protocol of the current location to request the image (necessary for development)

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

@CartoDB/frontend: can you have a look? I'm unsure about the location of the static map rendering

If we add more functionality it might be motivated to have individual views for the map preview, but they are after all only previews and not full-feature cards so I think where you placed it is fine for now.
Btw, why not use attribute like data-vizjson-url to avoid invalid attribute? (value can be accessed through .data method, i.e. $(this).data("vizjson-url"),)

Yeah, I agree with you that at some point it will make sense to have proper views.
I'll change that attribute name before merging the PR, thanks!

don't forget to update NEWS

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)


[WIP] Print a hint/warning towards necessary feature gates when importing modules

This works nicely for things like std.regex, but I'm not really happy with the errors when the error is transitive like with std.random
Not quite sure how to best do that without overcomplicating the build script though

Yeah that seems awkward :/

Seems good to me! What is the problem with errors in nested modules? That it is still possible to import! std.regex and then get an error about std.regex.prim ?

Yeah, the only way I can think of to make it better is to modify the import macro to allow you to disable some of the std modules. Everything else would be messing around with the build script more than I like.

Something like this: 7a0d372

There would essentially be a need to add substitution primitive modules with the error message for each file that uses the primitive module to make the error occur at the first imported module (so three modules for json ... https://github.com/gluon-lang/gluon/tree/master/std/json).
I don't think it is worth it, I have some patches to reduce the amount of cascading errors as well so ideally, the lack of say std.regex.prim shouldn't print out that many errors on top of the actual missing module error.

Yeah, I'll just remove the last commit real quick and then the rest should be ready to merge.

Yup, thanks!
bors r+


Configuration Manager URLs

Please document the Configuration Manager URLs that need to be whitelisted in the Proxy.
Example: https://blogs.technet.microsoft.com/configmgrdogs/2016/10/13/configuration-manager-proxy-exceptions/

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 926d0167-454c-f8c0-177d-0b1de31b8a77
Version Independent ID: e1a16df1-da95-3783-c683-cd80fe43bf6d
Content: Proxy server support - Configuration Manager
Content Source: sccm/core/plan-design/network/proxy-server-support.md
Product: configuration-manager
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Thanks @Stroell for the feedback. I'm going to close this as a duplicate of #714. That issue notes a few articles that already have some URLs listed


chore: remove duplicated zh-CN link

I was quite confused when I saw the duplicate link at the website.

Thanks!


Support upload data interception

I haven't written tests yet, although the existing tests pass.
(Separately, we might want to start using async stream reading, but that's a different matter.)

Removed the TODO around understanding requirements, and added a test for interception.

Thanks - will wait until the Travis fix is done, so I can merge on green. (It's not super-urgent, as I'm hoping to get download interception in before the next release anyway.)


exception handler intercepts c++ exceptions in the host on windows

embedding wamr into c++ application, after initializing, the wamr spams into console with every c++ exception thrown (in the host), even if completely unrelated to wasm.
the logging happens on https://github.com/bytecodealliance/wasm-micro-runtime/blob/c85bada2a9c85bc80f4c1a6baa7cefb662013eb4/core/iwasm/common/wasm_runtime_common.c#L386
my understanding is that this handler is used for HW bounds checks, in which case it should ignore all other types of exceptions, and only log when actual issue has happened.

So how about LOG_ERROR for the actual wasm exception and LOG_WARNING/LOG_VERBOSE to replace the log you are pointing out? And maybe add extra info like "unrecognized exception for wasm, pass to user's host exception handler" to explain the intention of return EXCEPTION_CONTINUE_SEARCH is to search for the next handler(transfer to the user's own handler).  What do you think?

Hi, I have looked into the code in more detail now.
I would prefer to log a warning when inside wasm, and unknown exception code only. There is no point in logging anything if the exception is not related to the wasm, as such cases should be handled in other handlers. And no logging is necessary when the exception code is handled with wasm_set_exception.
diff --git a/core/iwasm/common/wasm_runtime_common.c b/core/iwasm/common/wasm_runtime_common.c
index c6e9f0b9..3dbcc76d 100644
--- a/core/iwasm/common/wasm_runtime_common.c
+++ b/core/iwasm/common/wasm_runtime_common.c
@@ -384,12 +384,16 @@ runtime_exception_handler(EXCEPTION_POINTERS *exce_info)
                 return ret;
         }
 #endif
+        else {
+            LOG_WARNING("Unhandled exception thrown: "
+                        "exception code: 0x%lx, "
+                        "exception address: %p, "
+                        "exception information: %p\n",
+                        ExceptionRecord->ExceptionCode,
+                        ExceptionRecord->ExceptionAddress, sig_addr);
+        }
     }

-    LOG_ERROR("Unhandled exception thrown:  exception code: 0x%lx, "
-              "exception address: %p, exception information: %p\n",
-              ExceptionRecord->ExceptionCode, ExceptionRecord->ExceptionAddress,
-              sig_addr);
     return EXCEPTION_CONTINUE_SEARCH;
 }
 #endif /* end of BH_PLATFORM_WINDOWS */

Furthermore, It would be nice to have similar logging on linux, which currently does not log anything. However, I dont know if it is allowed to do so inside signal handler?

I think the current method in Linux is fine. If you look at the code logic here: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/shared/platform/common/posix/posix_thread.c#L610-L643
If the signal handler of the runtime cannot handle it, the runtime will first check whether there are other signal handlers registered previously. If there are, it will be transferred to it. If not, it will finally log it.

awesome. thanks ;)


createOrUpdateGraphNode functionality

Hi! I was trying to use createOrUpdateGraphNode but i get an exception () because i pass some fields that are not database fields, would it be okay to use a "graph fillable" variable in the model and modify mapGraphNodeFieldNamesToDatabaseColumnNames function?
---(In Model)  $fb_graph_fillable = ['name', 'email', 'facebook_id'];

public static function mapGraphNodeFieldNamesToDatabaseColumnNames(Model $object, array $fields)
{
    foreach ($fields as $field => $value) {
        if ( in_array(static::fieldToColumnName($field), static::$fb_graph_fillable) ) {
          $object->{static::fieldToColumnName($field)} = $value;
        }
    }
}

OR maybe...
public static function mapGraphNodeFieldNamesToDatabaseColumnNames(Model $object, array $fields)
{
  if (sizeof(static::$fb_graph_fillable) > 0){   /// if fb_graph_fillable has items, then filter
      foreach ($fields as $field => $value) {
           if ( in_array(static::fieldToColumnName($field), static::$fb_graph_fillable) ) {
                 $object->{static::fieldToColumnName($field)} = $value;
           }
       }
   }else{ /// else, dont filter, map all values
        foreach ($fields as $field => $value) {
             $object->{static::fieldToColumnName($field)} = $value;
        }
    }
}

Thanks for the idea! This feature will be added to the next tag. :)


Base64QRCode does not exist in 1.4.3

Type of issue
[x] Bug
[ ] Question (e.g. about handling/usage)
[ ] Request for new feature/improvement
Expected Behavior
No build errors after minor upgrade of 1.4.2 to 1.4.3
Current Behavior
Upgrading my nuget package dependency from 1.4.2 to 1.4.3 caused build errors in my project, because Base64QRCode could not be found
Possible Solution (optional)
Looks to be related to NET6_0_WINDOWS directives that have been added in this release
Steps to Reproduce (for bugs)

.Net 6 Web App.
Add 1.4.2 QRCoder package
Instantiate a Base64QRCode
Compile / Run and observe no build error
Upgrade to 1.4.3 package
Attempt to build solution. Compiler error: The type or namespace name 'Base64QRCode' could not be found (are you missing a using directive or an assembly reference?)

Your Environment
TargetOS of Web App: None.
By explicitly setting this to Windows, the compiler error is fixed, but this breaking change shouldn't be introduced in a patch level version change.

QRCode does not exist tooooooooooooooooooooooooooooooooo

Just use
Convert.ToBase64String(qrCodeByte)

QRCode does not exist tooooooooooooooooooooooooooooooooo

use specific constructor, such as PngByteQRCode(), AsciiQRcode(), BitmapByteQRCode

Hello @ScottyLightnin ,
this is not bug in QRCoder but more the consequence of breaking changes in Microsoft .NET 6.0.
Explanation - Why are some renderers missing
Up to QRCoder 1.4.2 there was no explicit compilation target for .NET6, thus when apps (that target .NET 6.0) used QRCoder, the package manager/Visual Studio picked the .NET 5.0 binaries when choosing the nearest framework binary. The .NET 5.0 binaries support System.Drawing and thus contain rendering classes like the Base64QRCode which makes use of System.Drawing.
Unfortunately not having .NET 6 as target in QRCoder lead to some serious trouble (see #354, #353) because not everything that works in .NET 5.0 also works in .NET 6.0 or in other words not all binaries/libraries to be available in .NET 5.0 can be expected available on .NET 6.0. Thus it was only consequent to add .NET 6.0 as target frameworks. I think you're right up to this point?
With adding net6.0 and net6.0-windows as targets, I had to remove the System.Drawing-based for the target net6.0, because Microsoft official says, that System.Drawing from .NET 6.0 on only will be available on Windows platforms: https://docs.microsoft.com/en-us/dotnet/core/compatibility/core-libraries/6.0/system-drawing-common-windows-only
TL;DR: It was not my decision to remove the System.Drawing-based renderers from net6.0 target, but I was simply following the breaking changes in .NET 6.0/System.Drawing (as described in the link above).
What can you do now?
You have multiple options:

Target your application to net6.0-windows as the renderes like Base64QRCode or QRCode are available there.
Use PngByteQRCode or BitmapByteQRCode. They return the QR Code image as byte array. Then simply pass the bytes to Convert.ToBase64String(...) to receive the same output as for Base64QRCode.
Stick with QRCoder 1.4.2 which hasn't the targets for net6.0 and thus via fallback to net5.0 allows you to use Base64QRCode. (The only difference between 1.4.2 and 1.4.3 are the new net6.0 targets as can be seen here: https://github.com/codebude/QRCoder/wiki/Release-notes )

Mid-term solution
Mid-term we will move QRCoder from System.Drawing to ImageSharp, because System.Drawing became a pain in the *** over the years. By replacing it with an alternative graphics lib (currently we aim to ImageSharp) we hope to gain wider support for the different platforms.
Note: Since this will bring a lot of breaking changes, this will be released as QRCoder 2.X.X

Afterthought: I stand by the above changes. The only point where I can understand your displeasure @ScottyLightnin is that these (breaking) changes came with version 1.4.3 and not with a 2.0.0. It should be noted that the QRCoder has always increased the version number by one digit with each release (see here). According to strict/clean semantic versioning the 1.4.3 should have been a 2.0.0. I can see that. For the future (also with regard to QRCoder 2.X.X) I plan to increase the version numbers cleanly/according to semantic versioning.

Great explanations, thank you @codebude , and yes, you got my main point was that I wasn't expecting a breaking change with the version change from 1.4.2 > 1.4.3. I've already been able to work around the problem (by targeting  net6.0-windows), it was just unexpected since I've grown accustomed to semantic versioning.
Thanks again for your work, and your detailed explanation. Much appreciated.

Having breaking changes in a minor patch seems strange to me to be honest, I had no idea why my code stopped working when I updated from 1.4.1 to 1.4.3.

How to pass foreground and background now with PngByteQrCode when we get graphic?

@mhosman either stick to QRCoder 1.4.2 since 1.4.3 added only the explicit targets to net 6.0 or wait for QRCoder 2.0.0 which will be based on ImageSharp and thus support the full renderer stack again.

Any time frame for version 2.0

On thing that I don't understant.... I'm working with 1.4.1 and .Net Core 6 in docker, and as long as I use this:
FROM base AS final
WORKDIR /app
RUN apt-get update && apt-get install -y libgdiplus
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "OfficeControl.React.dll"]

Looks like System.Drawing is workin OK in a linux environment, as long as I just suppress the warning:
#pragma warning disable CA1416 // Validate platform compatibility
            Graphics a = Graphics.FromHwnd((IntPtr)2);
#pragma warning restore CA1416 // Validate platform compatibility

On thing that I don't understant.... I'm working with 1.4.1 and .Net Core 6 in docker, and as long as I use this:
FROM base AS final
WORKDIR /app
RUN apt-get update && apt-get install -y libgdiplus
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "OfficeControl.React.dll"]

Looks like System.Drawing is workin OK in a linux environment.
If I use System.Drawing directly you need to supress the warning, but that's ok for me:
#pragma warning disable CA1416 // Validate platform compatibility
            Graphics a = Graphics.FromHwnd((IntPtr)2);
#pragma warning restore CA1416 // Validate platform compatibility

@olmobrutall that works like you can see here because you are installing libgdiplus in your linux environment.
What I am missing is how it is working without having set System.Drawing.EnableUnixSupport to true like written here or do you have this configuration switch activated?

FYI: https://security.snyk.io/vuln/SNYK-DOTNET-SYSTEMDRAWINGCOMMON-3063427

I would stay away from ImageSharp as that library now has licensing fees in certain environments:
https://github.com/SixLabors/ImageSharp/blob/main/LICENSE
A better alternative would be SkiaSharp or Magick.NET

Great library. Can we expect aforementioned mid-term solution to replace the System.Drawing?

Fixed by:

#495


Add a line number parameter for NotebookEditor.revealRange

I am working on improving the Live Share follow feature. Right now, we use NotebookEditor.revealRange to let one guest follow another throughout the workspace. However, this does not work if a cell is larger than the screen. When this happens, we reveal the range, but the viewport goes to the top of the cell, which may not be where the other guest is editing. To fix this, we need to add the line number to the reveal range function, so we can reveal the specific part of the cell that we need to, rather than just the top of the cell. Also, we need the onDidChangeNotebookEditorSelection to include the line number of the cell so we can track this.
cc @rebornix @fubaduba

cc @mjbvz

Hey @mjbvz and @rebornix!
We're looking at the E2E flow of collaborating Python and Jupyter notebooks for the Build 2022 keynote demo (timelines for which are late April). For those reasons, we would love to be able to prioritize this flow- where a user following another user during a Live Share session, is 'sent' to the 'right' place in the cell being edited, rather than the current default.
Could either of you provide us an estimate of when this work will be scheduled, and how if our team can provide any further clarity here to move it along?

@fubaduba We'd take a PR for this. The change would be adding an optional editorRange parameter to NotebookEditor.revealRange:
https://github.com/microsoft/vscode/blob/111b58221cbba5c9e3c2a6795f62aa85c2022bce/src/vscode-dts/vscode.proposed.notebookEditor.d.ts#L64

@mjbvz do you know if this is still needed for Live Share?

Closing for now. If live share still needs this, we can revisit


Re-enable watchdog and install debug version of the binary

This fixes https://jira.arm.com/browse/PELEDGE19-935.

We should not stop watchdog after starting it.
Install debug version of deviceos-wd for debugging purposes.
Reduced the "wait before starting hardware watchdog" time from 3000 to 300 aka 5 mins
Reduced the "The max allowed keep alive" time from 900 to 90 aka 1.5 mins

@ygoyal18 I thought rp100 was a completely different hardware device, and not a RPi.  Is that just a typo in the commit message, or will that same .so work for this device?  Or is it not device-specific?  Sorry, I'm a little confused about the naming here.

The rallypointwatchdog library implements the protocol to talk to deviceOSWD. That library is agnostics of the hardware, the naming of that library is actually confusing. rp100wd.so will work on every platform we support. deviceOSWD actually implements the hardware specific watchdog controller and hence abstracts out the hardware specific details.

Closing this PR as we are going to shelf deviceoswd on RPi platform. Originally that service was designed to overcome some of the hardware limitations of rallypoint series.


about wrong file names

due to the wrong extensions on "generate_reports.py" and "report.py", we were unable to see the power dist. outputs in the "metrics.csv" and lvs summary in the "manufacturability.rpt".

just realized that even though it fixes the problem with the number of insterted diodes, critical path, and manufact. report, it still cannot give any output about power reports and I believe this is because we do not have any power report with "Slowest/Typical/Fastest Corner" in "rcx_sta.power.rpt" or any file so the if statement in number 364 is kinda broken and it causes the regex in the line 372 to never execute.


Switch to using updated number instead of current for ds rollout

What?

Fixes bug : https://github.com/Shopify/kubernetes-deploy/issues/288
Switch from using currentNumberScheduled to updatedNumberScheduled since that is more important to judge correct success condition
Add unit tests that makes sure the deploy doesn't pass until the updated and desired numbers are equal in the status

Fixed the comments :)


TPS = 0, ticking block spam console (tube in IC2)

no logs , only error in console spam and 0 tps in server very lagged.

dupe #7


Remove all references to old dm-verity boot tooling

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[x] The toolchain has been rebuilt successfully (or no changes were made to it)
[x] The toolchain/worker package manifests are up-to-date
[x] Any updated packages successfully build (or no packages were changed)
[x] Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
[x] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[x] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[x] All source files have up-to-date hashes in the *.signatures.json files
[x] sudo make go-tidy-all and sudo make go-test-coverage pass
[x] Documentation has been updated to match any changes to the build system
[x] If you are adding/removing a .spec file that has multiple-versions supported, please add @microsoft/cbl-mariner-multi-package-reviewers team as reviewer (Eg. golang has 2 versions 1.18, 1.21+)
[x] Ready to merge

Summary

Breaking up PR into parts:

https://github.com/microsoft/azurelinux/pull/10973
https://github.com/microsoft/azurelinux/pull/10974
https://github.com/microsoft/azurelinux/pull/10975

Remove the now pointless parts of https://github.com/microsoft/azurelinux/pull/549 (since https://github.com/microsoft/azurelinux/pull/9537 removed dm-verity support from the imager tool).
Some parts of #549 were general refractors which are still needed, but everything related to building an image with dm-verity enabled has been removed.
See #9537 for details on migration paths using the imagecustomizer tool
Change Log

Remove ReadOnlyVerityRoot config struct
Remove verity-read-only-root package
Remove logic around verity roots in the toolkit

Does this affect the toolchain?

NO
Associated issues

https://microsoft.visualstudio.com/DefaultCollection/OS/_workitems/edit/51951945

Test Methodology

https://dev.azure.com/mariner-org/mariner/_build/results?buildId=670454&view=results

I've rebased this to the current state of the tools.

Parts merged in other prs


Enhance log file stats

Add more data into the LogFileStats class that breaks down counts by logical groupings to help in-IDE surveys determine which flutter channels users have used recently
Addresses:

https://github.com/dart-lang/tools/issues/53

[x] I’ve reviewed the contributor guide and applied the relevant portions to this PR.

Contribution guidelines:

See our contributor guide for general expectations for PRs.
Larger or significant changes should be discussed in an issue before creating a PR.
Contributions to our repos should follow the Dart style guide and use dart format.
Most changes should add an entry to the changelog and may need to rev the pubspec package version.
Changes to packages require corresponding tests.

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

@devoncarew just to confirm, i have labeled the version in this package with a wip prefix.. if i were to merge this in today, will there be bots that automatically update the revision in the sdk DEPS file?

@devoncarew just to confirm, i have labeled the version in this package with a wip prefix.. if i were to merge this in today, will there be bots that automatically update the revision in the sdk DEPS file?

Yes (two things):

we will sync this to the sdk; it's a manual (batch) process that we run ~twice a week. It syncs every dart package repo into the sdk from their head commits. You can stop the sync for an individual package via adding a comment into the SDK's DEPS file but that should be an infrequent occurance
wrt publishing - none of our automation will try to or offer to publish a -wip version

See https://github.com/dart-lang/sdk/blob/main/DEPS#L112 for how to temporarily disable syncing a repo.


TM-100: Add prettier to BE

Closes TM-100. Uses pre-commit hook to execute eslint and prettier

Done, also written in README


Custom consumer ID?

Right now the consumer generates its own ID (newGUID) because the customID field in ConsumerConfig is private.  Could you modify it so I can manually set the consumer ID in ConsumerConfig?
Awesome library, BTW.

...or use the client ID.

Any special reason for that? I don't know much about consumer IDs, except
that they must be unique for each consumer process/thread. Client ID
wouldn't work for that purpose, as a client might serve multiple consumers.
On 5 Feb 2015 00:23, "Sean Bowman"<EMAIL_ADDRESS>wrote:

...or use the client ID.
—
Reply to this email directly or view it on GitHub
https://github.com/bsm/sarama-cluster/issues/4#issuecomment-72970343.

My thought is that it can make it easier to track installations in larger systems using tools like Puppet and Chef.  You can coordinate the server names directly and match them up with what's in Zookeeper.

Sure, but you can run many consumers (groups) on one server. You can even run multiple consumers in a single process in multiple goroutines. I am a little afraid of exposing this to users as bad and really unexpected things can happen if two consumers share the same ID.
The kafka documentation suggests:

... each consumer is given a transient, unique consumer_id (of the form hostname:uuid) for identification purposes ...

I could expose a configurable consumerID prefix which defaults to hostname, would that be an option?

A configurable prefix would be a perfect compromise.

Please let me know if #40274b35626a468189d2135f654afb5a07cb584e is what you are looking for

Looks great.  Thanks for adding this.


Allow re-init on same stream in cudamallocasync allocator.

This fixes a bug in which destroying and re-creating a session causes the cuda_malloc_async allocator to fail.
For example, when TF_GPU_ALLOCATOR is set to cuda_malloc_async without this patch
import tensorflow as tf
tf.compat.v1.disable_eager_execution()
for i in [0, 1]:
    print("i={}".format(i))
    with tf.compat.v1.Session() as sess:
        pass

results in the error
F tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:390] Trying to set the stream twice. This isn't supported.

@hawkinsp Is the error above the same that you saw with real model in JAX with cudaMallocAsync?

@nouiz No, it wasn't. That symptom was memory corruption (missing synchronization?)

@nouiz No, it wasn't. That symptom was memory corruption (missing synchronization?)

Ok. I was taking a chance.


service instance is not created so client id and secret is unavailable

Tutorial URL: https://developers.sap.com/tutorials/cp-starter-integration-cpi-design-iflow.html
Step 11: Test integration flow using Postman
No idea as from where we have to copy the Client ID and Client Secret as we are not creating any service instances.(worked with trial user name and password)
Please include a step to create a service instance.
Labels:
Landscape: trail1, trail2, trail3
Label: SCPTest-2010A
Scenario Label: SCPTest-cloudin
Priority: High
Best Regards,
Priyanka

Hi Priyanka,
Thanks for the feedback.
The concern has been fixed from our end and the tutorial has been updated in the production.
Could you please close this issue from your end?
Regards,
Vikram Kulkarni


add ADR-33 consistant metadata to micro service and endpoints

We had some feedback in the new tech previews channel and ADR-33 came later than micro ADR so seems good to have this for consistency.
The use case is that these annotations can be read to build monitoring, ingress and more in a zero config manner.

Go PR adding support for this https://github.com/nats-io/nats.go/pull/1231 hope it clarifies

I think we need to add metadata also to endpoint stats, today its only in schema but in order to build a UI that just shows info about a service we already need to call info (no endpoint details just subjects) and then stats to get endpoint details. As is we would also need to call schema to get endpoint metadata and then a bunch of work to combine this information.
So before I send a updated PR just wanted to solicit feedback on that @piotrpio @aricart

@ripienaar the funny thing is - this is what I already did - when you call stats, the metadata is included. I think schema shouldn't be part of that, as that may be rather verbose.
The part I need to verify on my PR is that schema is including metadata as well.

@ripienaar the funny thing is - this is what I already did - when you call stats, the metadata is included. I think schema shouldn't be part of that, as that may be rather verbose.
The part I need to verify on my PR is that schema is including metadata as well.

You need to be careful about just adding random stuff as the CLI will schema validate responses :)


Python package problem

Hi,
Thank you for the great simulator! I am trying to use the autodrive ros2 api, but get this problem when I try to install the python packages:
marno@arragon:~/projects/AutoDRIVE/ros2_ws$ pip3 install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: eventlet==0.33.3 in /home/marno/.local/lib/python3.10/site-packages (from -r requirements.txt (line 1)) (0.33.3)
Requirement already satisfied: Flask==1.1.1 in /home/marno/.local/lib/python3.10/site-packages (from -r requirements.txt (line 2)) (1.1.1)
Requirement already satisfied: Flask-SocketIO==4.1.0 in /home/marno/.local/lib/python3.10/site-packages (from -r requirements.txt (line 3)) (4.1.0)
Requirement already satisfied: python-socketio==4.2.0 in /home/marno/.local/lib/python3.10/site-packages (from -r requirements.txt (line 4)) (4.2.0)
Requirement already satisfied: python-engineio==3.13.0 in /home/marno/.local/lib/python3.10/site-packages (from -r requirements.txt (line 5)) (3.13.0)
Collecting greenlet==1.0.0
  Using cached greenlet-1.0.0.tar.gz (84 kB)
  Preparing metadata (setup.py) ... done
Collecting gevent==21.1.2
  Using cached gevent-21.1.2.tar.gz (5.9 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [57 lines of output]
      Compiling src/gevent/resolver/cares.pyx because it changed.
      [1/1] Cythonizing src/gevent/resolver/cares.pyx
      performance hint: src/gevent/libev/corecext.pyx:1325:5: Exception check on '_syserr_cb' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.
      Compiling src/gevent/libev/corecext.pyx because it changed.
      [1/1] Cythonizing src/gevent/libev/corecext.pyx
      Compiling src/gevent/_greenlet_primitives.py because it changed.
      [1/1] Cythonizing src/gevent/_greenlet_primitives.py
      Compiling src/gevent/_hub_primitives.py because it changed.
      [1/1] Cythonizing src/gevent/_hub_primitives.py
      Compiling src/gevent/_hub_local.py because it changed.
      [1/1] Cythonizing src/gevent/_hub_local.py
      Compiling src/gevent/_waiter.py because it changed.
      [1/1] Cythonizing src/gevent/_waiter.py
      warning: src/gevent/_gevent_cgreenlet.pxd:112:33: Declarations should not be declared inline.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      cdef load_traceback
      cdef Waiter
      cdef wait
      cdef iwait
      cdef reraise
      cpdef GEVENT_CONFIG
            ^
      ------------------------------------------------------------

      src/gevent/_gevent_cgreenlet.pxd:182:6: Variables cannot be declared with 'cpdef'. Use 'cdef' instead.
      Compiling src/gevent/greenlet.py because it changed.
      [1/1] Cythonizing src/gevent/greenlet.py
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 267, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 294, in <module>
          EXT_MODULES.append(cythonize1(mod))
        File "/tmp/pip-install-axxodj9m/gevent_188e47a78e0a4647b09609828c5fac26/_setuputils.py", line 237, in cythonize1
          new_ext = cythonize(
        File "/tmp/pip-build-env-p730dgbz/overlay/local/lib/python3.10/dist-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
          cythonize_one(*args)
        File "/tmp/pip-build-env-p730dgbz/overlay/local/lib/python3.10/dist-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: src/gevent/greenlet.py
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Inside requirements.txt:
eventlet==0.33.3
Flask==1.1.1
Flask-SocketIO==4.1.0
python-socketio==4.2.0
python-engineio==3.13.0
greenlet==1.0.0
gevent==21.1.2
gevent-websocket==0.10.1
attrdict
numpy
pillow
opencv-contrib-python

I'm running Ubuntu 22.04 with ROS2 Humble

Hi,
This seems to be a dependency issue, which we haven't come across before. That being said, we are happy to debug it together with you.
After searching for a similar issue, we found this issue, which states that Python 3.11 is missing certain pre-built wheels required by gevent. In case that is indeed the issue, the solution to another issue here mentions running the following command before installing gevent: apt-get update && apt-get install -y --no-install-recommends build-essential
Can you let us know your Python version? Alternatively, you could create a virtual environment with a slightly older Python version (we have tested with Python 3.8.10 and 3.9.13) to see if that helps.
Let us know how it goes. Best of luck!

Hi @Marnonel6 , seems like you're using python 3.10 and as @Tinker-Twins mentioned, that might be the issue.
I was in the same boat because I was running Devkit on 3.10.12.
Here's the workaround for this issue:
The primary reason for your error is gevent's version not supporting the newer Cpython's recynthonizing. See https://github.com/gevent/gevent/issues/1899#issuecomment-1270737391
The workaround is to set gevent to a newer version. In my case, I followed the suggestion in https://github.com/gevent/gevent/issues/1899#issuecomment-1264352460 and changed to pip install gevent>=21.12.0.
After that, you'll encounter another problem in which there's an import error on the line from collections import Mapping from attrdict because of attrdict's deprecation in >=python3.10.
For that problem, I replaced all instances of from attrdict import AttrDict with the monkey patch here. It essentially copies attributes from collections to collections.abc.
@Tinker-Twins I tried creating a virtual environment by installing Python 3.8 that's callable by python3.8 and creating a virtualenv with it. However, the sourcing is complicated with building the package, sourcing the built package, etc. to the point where the same error persists because the python3 at my /usr/bin/env was still used in the package, which is version 3.10.12.


Single gpu

Why are you using only a single GPU?
If you use DistributedDataParallel or DataParallel, does it slow down?

Due to limitations in laboratory computing resources at the time, I only used one GPU. It is feasible to use multiple GPUs during training.


In case of Proxy Error, try again

Was not able to test this as I did not got the Proxy Error again, but seems to be OK for normal usage

This will patch only a single place where queries happen. There are many of them and all need to be patched.

Again, thank you very much!


Add support for "PSQL Stored Functions" and "Packages"

Firebird 3.0 introduced 2 new concepts:

PSQL Stored Functions (See:  https://firebirdsql.org/file/documentation/release_notes/html/en/3_0/rnfb30-psql-funcs.html)
Packages (See: https://firebirdsql.org/file/documentation/release_notes/html/en/3_0/rnfb30-psql-packages.html)

Please Add support for "PSQL Stored Functions" and "Packages" by:

Creating the Packages node;
Ability for creating, altering, etc "PSQL Stored Functions" and "Packages"

Hi @luronumen
Can you give me feedback on this or close it?
Thanks
./jo

For now we can close it!


Coverage calculation

This is draft. We need this to be merged first, as we want to use master git branch for dependency.

@jawoznia yes, coverage measurement works but a little bit differently than you could expect.
There are three coverages you can consider here:

The test coverage - it shows you what is coverage of your test code. Basically, if you create a contract in the test, you measure what is the coverage of your custom contract code. So it is telling you how many of the features used in this test case are actually tested.
The generated code coverage - how many of the code generation cases are covered in tests? 100% means, that all code which would be generated is generated somewhere in tests (but it still may be not tested at all). That means that this coverage on its own is useless - it has to be paired with 1). And we want to be as close to 100% as physically possible. The actual coverage could be probably estimated as 2) * 1).
Contracts coverage, being just examples coverage - nice to improve here, but not critical. We probably won't bother unless we provide MT utils generation.


feat: start counter with dummy numbers

Description
Add a temporary counter to the website that counts up the days since Danaher promised an audit.
Type of change

[x] New content/feature (new page, new section, etc.)

Misc Notes
preview(s):

https://tbfighters.a-good.dev/
https://tbfighters.a-good.dev/audit

Do not merge until this tuesday


chore: Create query selector

This PR updates the WidgetQueryGeneratorForm to only allow queries so that it looks like a query selector. This is needed for chat widget where the widget can only bind Appsmith AI queries.

PS: The above image is an example where TableWidget had allowedDatasourcestypes as "Mysql" ( just for testing )

Summary by CodeRabbit

New Features

Introduced a new dropdown variant "Connect to query" for improved datasource selection.
Enhanced handling of dynamic query options and datasource filtering based on context.

Improvements

Updated the Widget Query Generator Form to support new properties for better data management.
Improved control flow in dropdown options to ensure appropriate options are displayed based on the current context.
Enhanced the One Click Binding Control to utilize new properties for better interaction with the Widget Query Generator Form.

Bug Fixes

Refined logic for displaying datasource options, preventing unnecessary options from appearing.

[!WARNING]
Tests have not run on the HEAD 58b3bae07692688ee7d74f29bac6a8fba8d97f99 yet
Wed, 23 Oct 2024 10:34:55 UTC

What happens if the Appsmith AI Datasource doesn't exist?
You will see an empty dropdown in that case. That state is properly handled in UI as of now. Btw, as soon as you drop the chat widget, we create a new Appsmith AI query ( @KelvinOm do we also create datasource if it does not exists? )

if the user wants to use their own datasource?
This case won't happen. the AIChat widget explicitly require AppsmithAI datasoruce cause that datasources is handling lot of stuff under the hood. So, it can't be binded with other datasources like MySQL, postgres.

The way the widget tells that it only needs AppsmithAI is by specifying in the propertyPaneConfig like this:
{
	propertyName: "chatQuery",
	helpText: "The query to be used for the chat",
	label: "Chat Query",
	controlType: "ONE_CLICK_BINDING_CONTROL",
	controlConfig: {
	  showEditFieldsModal: false,
	  datasourceDropdownVariant: DROPDOWN_VARIANT.CONNECT_TO_QUERY
	  excludePrimaryColumnFromQueryGeneration: true,
	  isConnectableToWidget: false,
	  allowedDatasourceTypes: [PluginName.APPSMITH_AI], // <-------------------
	  getQueryBindingValue: (query: ActionData | ModuleInstanceData) => {
	    return query.config.name;
	  },
	},
}

do we also create datasource if it does not exists?

@jsartisan  Yes, if datasource is not exists, then we create it, if there is one, we add query to it.


Ability to handle ties

Should automatically advance additional teams if there is a tie

Fixed in PR #7


Example not working

Hey! Thanks for the library! I'm not sure why, but I'm not getting any output from your example: https://play.golang.org/p/y8zs89hhzfs

What version of Go do you use?

Haven't tried it locally, only on playground which is running version 1.13.7

It is probably the playground failing because this package is quite hefty on the resources


Trimmed down features of CIMultiDict

Should result in a 8% speed boost

Just so I'm clear, since I submitting the original request: some user agents pass the UA string as "user-agent" and some as "User-Agent."
With CIMultiDict, I could use either spelling. Do I have to use the lower case version after this change?

No casefold is case-insensitive. @jasonab

Doesn't this break the case where multiple headers of the same name get sent? This is probably a pretty rare scenario but it is possible and seems to be the point of using something like MultiDict in the first place as it can handle those types of values. Relevant RFC section.
Here is an example of sending multiple headers of the same name with curl:
$ curl localhost:8000 -H 'Foo:bar' -H 'Foo:baz' --verbose
* Rebuilt URL to: localhost:8000/
*   Trying ::1...
* connect to ::1 port 8000 failed: Connection refused
*   Trying <IP_ADDRESS>...
* Connected to localhost (<IP_ADDRESS>) port 8000 (#0)
> GET / HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.43.0
> Accept: */*
> Foo:bar
> Foo:baz
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 17
< Connection: keep-alive
< Keep-Alive: timeout=60
<
* Connection #0 to host localhost left intact
{"hello":"world"}

If the view for this I have the following:
print(request.headers.getall('foo'))

which results in the following getting displayed:
['bar', 'baz']

There is no way to do this now after this change.

I would like to reiterate the concern raised by @awiddersheim above.
This is no longer a CIMultiDict, it is just a CIDict. It breaks any case where the server should be able to receive multiple of the same header, or send a reply with multiple of the same header.

I just found another issue with this implementation.
Calling my_headers.update() or my_headers.setdefault() both do not convert the key to a case invariant form.
For example:
my_headers = CIDict()
my_headers['Foo'] = 'Bar' # <-- This converts 'Foo' to 'foo' on `__setitem__`
my_headers.update({'Foo': 'Baz'}) <-- This does not call `__setitem__` so key remains 'Foo'

Expected actual contents of my_headers is:
{'foo': 'Baz'}

However actual contents is:
{'Foo': 'Baz', 'foo': 'Bar'}

This is because both update() and setdefault() methods on a dict call PyDict_SetItem() directly in C.
See this stackoverflow for more information http://stackoverflow.com/questions/2060972/subclassing-python-dictionary-to-override-setitem
You need to provide override implementations of update() and setdefault() which either call __setitem__ or do the casefold itself.


[SP-3514] Backport of PDI-16178 - arrayIndexOutOfBoundsException when collapsing a folder in "Open File" dialog (7.1 Suite)

Build Completed
:white_check_mark: This pull request has passed all validations.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f ui/build.xml clean-all resolve jacoco && ant -f ui/build.xml checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  ui/src/org/pentaho/di/ui/repository/dialog/SelectObjectDialog.java

@mbatchelor @pamval @mchen-len-son please review


Handle custom type constructors starting with non-ASCII letters

Fixes #78.

Thank you both! The fix has been released in v1.1.30 :relaxed: