PDO receive and transmit

Hello!
So i have been working really heavily in your stack through the Zephyr OS and I have loved it (so thank you firstly). So far I have been able to get a board up and running sending messages from the MCU and controlling different parts of the system via CANopen. But to further my understanding I started dabbling in PDOs and what they have to offer with their mapping capabilities and just larger data size they can transmit.
In an effort to confuse myself less, i used the canopen through Python to map and command my node as a "master". I was attempting to do that through your stack on an MCU instead of a Linux PC that I am currently using and I was a little lost by the PDO.c/.h files and their wording.
For reference here are the steps straight from my python script so I can describe exactly what I am doing:
`
network = canopen.Network()

network.connect()

node = network.add_node(NODEID, EDS)

node.nmt.state = 'PRE-OPERATIONAL'
time.sleep(1)

node.tpdo.read()
node.rpdo.read()

node.tpdo[3].clear()
node.tpdo[3].add_variable(0x6041)
node.tpdo[3].add_variable(0x606C)
node.tpdo[3].trans_type = 1
node.tpdo[3].enabled = True
node.tpdo.save()

node.rpdo[3].clear()
node.rpdo[3].add_variable(0x6040)
node.rpdo[3].add_variable(0x60FF)
node.rpdo[3].enabled = True
node.rpdo.save()

n = 6000

node.nmt.state = 'OPERATIONAL

node.sdo[0x6060].raw = 3
time.sleep(2)

node.rpdo[3][0x60FF].raw = n
node.rpdo[3].transmit()
time.sleep(1)

node.rpdo[3][0x6040].raw = 6
node.rpdo[3].transmit()
time.sleep(1)

node.rpdo[3][0x6040].raw = 271

node.rpdo[3].transmit()
time.sleep(1)

node.rpdo[3][0x6040].raw = 15
node.rpdo[3].transmit()
time.sleep(1)

while True:
	node.tpdo[3].wait_for_reception()
	print(node.tpdo[0x6041].raw)
	print(node.tpdo[0x606C].raw)
	node.rpdo[3][0x6040].raw = 7
	node.rpdo[3][0x60FF].raw = n
	node.rpdo[3].transmit()

	node.rpdo[3][0x6040].raw = 31
	node.rpdo[3].transmit()
	time.sleep(1)

	print("")
	n = int(n*0.8)

	if n <= 100:
		break

node.rpdo[3][0x6040].raw = 271
node.rpdo[3].transmit()

network.disconnect()

`
Ignoring the switch of NMT states (I am able to do that) and sending the SDO (i can also do that) I was wondering how to go about calling your functions to achieve this functionality. I know once I get the general order of your particular function calls this will be very easy to implement.
So in short, I initially map the TPDO and RPDO of the node at NODEID (for your reference this current setup has 8 TPDOs and 8 RPDOs). Once mapped/saved during PRE-OPERATIONAL state. I transition to OPERATIONAL state begin update the RPDO values of the node that I had just stored. Once i affect it, I just transmit it and move on to the next message i want to transmit.
Once i have successfully configured the 0x6040 to the state i need it to be in, I begin sending 60FF messages through that same transmit process. So very very basic (I really just want to understand how to imitate this here).
In another thread altogether I started sending SYNC messages:
That just starts sending sync message every second and stops it in about five minutes (300 seconds).
network.sync.start(1)
time.sleep(300)
network.sync.stop()

This so my TPDO transmits messages on the SYNC.
I also want to be able to send SYNC messages. My EDS currently has a value of 0x40000000 which i pulled from your check to set it as a producer but when i read back if the CO->SYNC->isProducer it returns a false
I look forward to your  response and thank you for your time I appreciate it!

To further add, just looking through the functions that are defined for the system, it seems as though it is handling the RPDO and TPDO of its own OD and not able to access or request to change the PDOs of the slave nodes. Maybe i am reading the docs and the code wrong but it seems that way to me, so those are my more major questions.
Ofcourse a method to do this is to map the RPDO of the Master to the TPDO of the Slave and the TPDO of the Master to the RPDO of the Slave but I want to also understand if there is a way to Map the PDOs on the slave prior to trying to access it.
Lastly, with the sync messages, there is a SYNC generator boolean that I cannot determine how to set on, is the 1005 value meant to be anything beyond the 0x40000000 value

To further add, just looking through the functions that are defined for the system, it seems as though it is handling the RPDO and TPDO of its own OD and not able to access or request to change the PDOs of the slave nodes. Maybe i am reading the docs and the code wrong but it seems that way to me, so those are my more major questions.
Ofcourse a method to do this is to map the RPDO of the Master to the TPDO of the Slave and the TPDO of the Master to the RPDO of the Slave but I want to also understand if there is a way to Map the PDOs on the slave prior to trying to access it.
Lastly, with the sync messages, there is a SYNC generator boolean that I cannot determine how to set on, is the 1005 value meant to be anything beyond the 0x40000000 value

Thank you too.
You really experiment quite a lot. CANopen may be hard to start, but once you understand the details, I think, it is easy. I will prepare some more examples, including PDO, in https://github.com/CANopenNode/CANopenSocket. There is something in "basicDevice", but no PDO yet.
First I would like to suggest you, to monitor raw CAN messages:
candump can0 #or something

Until you have heavy PDO communication, you can easily see, what is going on. You should understand each message:

bootup or heartbeat message with info about current NMT state
emergency message (some errors may prohibit NMT operational)
SDO messages are in pair(s)
PDO messages works only in NMT operational, contains data only, configured are by communication and mapping parameters,
Sync, etc.

Maybe I'm explaining things, you already know. I think, we should go step-by-step. And don't think about CANopen as master-slave protocol. More like producer-consumer(s). There are combinations.
These are some hints. Please include some candump, if you have questions, so I could easily see the problem.

Thank you too.
You really experiment quite a lot. CANopen may be hard to start, but once you understand the details, I think, it is easy. I will prepare some more examples, including PDO, in https://github.com/CANopenNode/CANopenSocket. There is something in "basicDevice", but no PDO yet.
First I would like to suggest you, to monitor raw CAN messages:
candump can0 #or something

Until you have heavy PDO communication, you can easily see, what is going on. You should understand each message:

bootup or heartbeat message with info about current NMT state
emergency message (some errors may prohibit NMT operational)
SDO messages are in pair(s)
PDO messages works only in NMT operational, contains data only, configured are by communication and mapping parameters,
Sync, etc.

Maybe I'm explaining things, you already know. I think, we should go step-by-step. And don't think about CANopen as master-slave protocol. More like producer-consumer(s). There are combinations.
These are some hints. Please include some candump, if you have questions, so I could easily see the problem.

Yes! I am glad you re-clarified because it never hurts to have some extra confirmation :)
I will send a candump from my setup in a few hours and I will explain the respective nodes and the ones you should pay attention to etc.
But I look forward to your PDO example as it really has me scratching me head at what to do.
There was also something that I noticed with the COB-ID that you set has the default value instead of the value that it reads in from the EDS file that I thought was interesting because i was trying to force setting the COB-IDs to equate to one another from the perspective of the TPDO matching the RPDO on the producer and vice versa on the consumer.
Ofcourse naturally, at least my understanding is that we "map" the PDO through a set of SDO commands very similar to how the SDO is intended to work.  So with that I would be interested to understand what that looks like to process the received data etc.
But I think everything will make more sense once i send over the candump!
Thank you!

Yes! I am glad you re-clarified because it never hurts to have some extra confirmation :)
I will send a candump from my setup in a few hours and I will explain the respective nodes and the ones you should pay attention to etc.
But I look forward to your PDO example as it really has me scratching me head at what to do.
There was also something that I noticed with the COB-ID that you set has the default value instead of the value that it reads in from the EDS file that I thought was interesting because i was trying to force setting the COB-IDs to equate to one another from the perspective of the TPDO matching the RPDO on the producer and vice versa on the consumer.
Ofcourse naturally, at least my understanding is that we "map" the PDO through a set of SDO commands very similar to how the SDO is intended to work.  So with that I would be interested to understand what that looks like to process the received data etc.
But I think everything will make more sense once i send over the candump!
Thank you!

I will try to explain PDO a little:
It is basically a raw CAN message with CAN-ID and up to 8 bytes of data. We have somewhere documented the meaning of data in specific PDO. So PDO is no more than CAN message, transmitted by one CAN(open) device and received by none, one or many CAN(open) devices.
CANopen gives us the rules, how we can pick some data, which are already inside (large) CANopen object dictionary (mapping parameters) and how to send or receive the PDO (communication parameters).
Pre-defined COB-IDs are often confusing. They are just a recommendation. Usually we use default value for TPDO. And for RPDO we use the COB-ID which matches the TPDO in interest. This is producer-consumers model. In master-slaves model pre-defined COB-IDs are more useful.
Example:

first device: our application writes data to OD (object dictionary) variable. Configure TPDO and map to that OD variable.
second device: configure RPDO to match above TPDO. Map data to custom OD variable, named valueFromFirstDevice. Application on second device reads that OD variable.

I will try to explain PDO a little:
It is basically a raw CAN message with CAN-ID and up to 8 bytes of data. We have somewhere documented the meaning of data in specific PDO. So PDO is no more than CAN message, transmitted by one CAN(open) device and received by none, one or many CAN(open) devices.
CANopen gives us the rules, how we can pick some data, which are already inside (large) CANopen object dictionary (mapping parameters) and how to send or receive the PDO (communication parameters).
Pre-defined COB-IDs are often confusing. They are just a recommendation. Usually we use default value for TPDO. And for RPDO we use the COB-ID which matches the TPDO in interest. This is producer-consumers model. In master-slaves model pre-defined COB-IDs are more useful.
Example:

first device: our application writes data to OD (object dictionary) variable. Configure TPDO and map to that OD variable.
second device: configure RPDO to match above TPDO. Map data to custom OD variable, named valueFromFirstDevice. Application on second device reads that OD variable.

Thank you! That makes a lot more sense in terms of the COB-ID i was overcompensating but in that case that clarifies things a lot more.
This is the candump form my terminal of running the code above. 701 is the CANnode running your stack, you can ignore it for now as it does nothing exciting in this demonstration. 719 is the node that gets mapped and sends messages one change. So from the code above i changed the PDO transmit type from 1 (which was every SYNC message) to 255 (to transmit on change of value which i figured is easier to track)
Let me know what you think!
candump.txt

Thank you! That makes a lot more sense in terms of the COB-ID i was overcompensating but in that case that clarifies things a lot more.
This is the candump form my terminal of running the code above. 701 is the CANnode running your stack, you can ignore it for now as it does nothing exciting in this demonstration. 719 is the node that gets mapped and sends messages one change. So from the code above i changed the PDO transmit type from 1 (which was every SYNC message) to 255 (to transmit on change of value which i figured is easier to track)
Let me know what you think!
candump.txt

It seems everything is OK. 719 sends no emergency message after bootup. In operational TPDOs seems to work.

It seems everything is OK. 719 sends no emergency message after bootup. In operational TPDOs seems to work.

Yes so this is what the python script at the top sends. I am wondering if this is imitable through your stack via the 701 node mapping the 719 node.

Yes so this is what the python script at the top sends. I am wondering if this is imitable through your stack via the 701 node mapping the 719 node.

I don't have much experience with CANopen for Python, but it enables nice interaction with CANopen network, as shown in your example. My stack have so called gateway interface with ascii commands, as shown in readme.md and in basicDevice example. These are two alternatives. To configure PDO for example, Python uses nice commands, without care what is happening in background. With ascii commands in my stack it is necessary to use raw SDO read/write commands.
There is also a third type of tools for CANopen: a GUI, where you can configure CANopen network with drag and drop, etc.
There are a lot of options around CANopen.
One of the advantages of CANopen network is, it can run without a master, once it is configured. Just exchanging PDOs and heartbeats.
Another advantage is, it can be very deterministic and real-time, better than 1 millisecond. Python may be slower at this point, but it depends on usage.
Do you like me to comment your candump.txt a little? If so, can you provide candump with timestamps:
candump -td -a can0 | tee candump2.txt

I don't have much experience with CANopen for Python, but it enables nice interaction with CANopen network, as shown in your example. My stack have so called gateway interface with ascii commands, as shown in readme.md and in basicDevice example. These are two alternatives. To configure PDO for example, Python uses nice commands, without care what is happening in background. With ascii commands in my stack it is necessary to use raw SDO read/write commands.
There is also a third type of tools for CANopen: a GUI, where you can configure CANopen network with drag and drop, etc.
There are a lot of options around CANopen.
One of the advantages of CANopen network is, it can run without a master, once it is configured. Just exchanging PDOs and heartbeats.
Another advantage is, it can be very deterministic and real-time, better than 1 millisecond. Python may be slower at this point, but it depends on usage.
Do you like me to comment your candump.txt a little? If so, can you provide candump with timestamps:
candump -td -a can0 | tee candump2.txt

Hello!
Apologies for the delay could not carve out enough time to get back to my setup!
I have attached the candump2.txt formed by running the python script and your above command.
To comment further, I completely agree! That is where i want to reach in terms of how I set up the system, once i send a set of SDOs to map a PDO, I just want to receive data from the node as it changes. I am interested to see what I need to do on your end to receive these mapped PDOs and retrieve  their data. But once you have time to generate an example that will help.
Thank you again, I look forward to your response and let me know your thoughts and if there is something I should consider as I transition this logic to your stack!
candump2.txt

Hello!
Apologies for the delay could not carve out enough time to get back to my setup!
I have attached the candump2.txt formed by running the python script and your above command.
To comment further, I completely agree! That is where i want to reach in terms of how I set up the system, once i send a set of SDOs to map a PDO, I just want to receive data from the node as it changes. I am interested to see what I need to do on your end to receive these mapped PDOs and retrieve  their data. But once you have time to generate an example that will help.
Thank you again, I look forward to your response and let me know your thoughts and if there is something I should consider as I transition this logic to your stack!
candump2.txt

I will try to explain candump2.txt. It shows actual CAN communication between two devices. This is example of master-slave type of communication.
line 2: (001.999987)  can0  719   [1]  7F                        '.'

Line 2 shows CAN message, occurred 2 seconds after previous CAN message. It is from can0 interface, which is probably USB to CAN interface connected to Linux computer, which behaves like a CANopen master in this example. Another device on CAN bus is microcontroller, running CANopenNode, I guess. It could be any CANopen device.
However, this second device has Node-Id = 0x19. Message above is Heartbeat message from this second device. CANopen heartbeat message is CAN message, with 11-bit identifier (0x700 + node-id) and has one data byte. For default 11-bit CAN identifiers used by CAN open see standard CiA301 (7.3.3 Generic pre-defined connection set) or CO_Default_CAN_ID_t from CANopenNode documentation.
Standard CAN messages all have 11-bit identifier and 0 to 8 data bytes. CAN message with lower identifier has higher priority on the bus. There are other types of CAN messages, not used here: 29-bit extended identifier; CAN FD, newer with more data and higher data rates, etc.
Heartbeat message contains one data byte, 0x7F in our case. It shows internal NMT state of the device, NMT pre-operational in our case. See NMT states in standard or CO_NMT_internalState_t
line 6: (000.248715)  can0  000   [2]  82 19                     '..'
line 7: (000.007631)  can0  719   [1]  00                        '.'
line 8: (001.999969)  can0  719   [1]  7F                        '.'

Line 6 is NMT command from master: CAN-id = 0 - highest priority message. CO_NMT_command_t 0x82 (reset communication) is sent to node-id=0x19. Node 19 responded with boot-up message and after two seconds with heartbeat. CANopenNode gateway (CiA309-3) produces the same NMT master message with command [1] 0x19 reset communication.
line 21: (000.481348)  can0  619   [8]  40 00 18 01 00 00 00 00<EMAIL_ADDRESS>line 22: (000.002445)  can0  599   [8]  43 00 18 01 99 01 00 00   'C.......'

This is SDO. Master (SDO client) sends read or write request for variable form Object Dictionary (CAN-ID = 0x600 + remote-node-id). SDO server responses (CAN-ID = 0x580 + node-id). SDO messages are harder to read, usually it is not necessary to know the details. For description see CiA301 or CO_SDO_state_t.
Line 21 is CO_SDO_ST_UPLOAD_INITIATE_REQ, index=0x1800, subindex=0x01
Line 22 is CO_SDO_ST_UPLOAD_INITIATE_RSP, same index and subindex, data=0x00000199
Equivalent CANopenNode gateway command: [1] 0x19 read 0x1800 1 x32.
line 25: (000.000710)  can0  619   [8]  40 00 18 03 00 00 00 00<EMAIL_ADDRESS>line 26: (000.002455)  can0  599   [8]  80 00 18 03 11 00 09 06   '........'

Line 25 is CO_SDO_ST_UPLOAD_INITIATE_REQ, index=0x1800, subindex=0x03
Line 26 is CO_SDO_ST_ABORT, same index and subindex, CO_SDO_abortCode_t = 0x06090011 #Sub index does not exist (this is strange, TPDO Inhibit time, Event timer and SYNC start value are missing)
This pattern then repeats and reads all TPDO and RPDO communication and mapping parameters. As in Python program.
line 183: (000.001001)  can0  619   [8]  23 00 18 01 99 01 00 80   '#.......'
line 184: (000.002439)  can0  599   [8]  60 00 18 01 00 00 00 00   '`.......'

Line 183 is CO_SDO_ST_DOWNLOAD_INITIATE_REQ, index=0x1800, subindex=0x01, data=0x80000199
Line 184 is CO_SDO_ST_DOWNLOAD_INITIATE_RSP, same index and subindex, OK
Equivalent CANopenNode gateway command: [1] 0x19 write 0x1800 1 x32 0x80000199 - disable PDO.
Then follows several similar commands configuring TPDOs and RPDOs.
After that NMT command sets node 0x19 to operational and then appears four different PDO messages on the bus.
Python is certainly most clear. More raw approach may also be with several CANopen gateway commands. Observing candump may be useful for understanding the details or for debugging.
I didn't decode all SDO commands and the rest of the program, but I hope I explained a little. It is important to know, where does TPDOs came from and what are the contents. See also some explanation of PDO re-mapping procedure.

I will try to explain candump2.txt. It shows actual CAN communication between two devices. This is example of master-slave type of communication.
line 2: (001.999987)  can0  719   [1]  7F                        '.'

Line 2 shows CAN message, occurred 2 seconds after previous CAN message. It is from can0 interface, which is probably USB to CAN interface connected to Linux computer, which behaves like a CANopen master in this example. Another device on CAN bus is microcontroller, running CANopenNode, I guess. It could be any CANopen device.
However, this second device has Node-Id = 0x19. Message above is Heartbeat message from this second device. CANopen heartbeat message is CAN message, with 11-bit identifier (0x700 + node-id) and has one data byte. For default 11-bit CAN identifiers used by CAN open see standard CiA301 (7.3.3 Generic pre-defined connection set) or CO_Default_CAN_ID_t from CANopenNode documentation.
Standard CAN messages all have 11-bit identifier and 0 to 8 data bytes. CAN message with lower identifier has higher priority on the bus. There are other types of CAN messages, not used here: 29-bit extended identifier; CAN FD, newer with more data and higher data rates, etc.
Heartbeat message contains one data byte, 0x7F in our case. It shows internal NMT state of the device, NMT pre-operational in our case. See NMT states in standard or CO_NMT_internalState_t
line 6: (000.248715)  can0  000   [2]  82 19                     '..'
line 7: (000.007631)  can0  719   [1]  00                        '.'
line 8: (001.999969)  can0  719   [1]  7F                        '.'

Line 6 is NMT command from master: CAN-id = 0 - highest priority message. CO_NMT_command_t 0x82 (reset communication) is sent to node-id=0x19. Node 19 responded with boot-up message and after two seconds with heartbeat. CANopenNode gateway (CiA309-3) produces the same NMT master message with command [1] 0x19 reset communication.
line 21: (000.481348)  can0  619   [8]  40 00 18 01 00 00 00 00<EMAIL_ADDRESS>line 22: (000.002445)  can0  599   [8]  43 00 18 01 99 01 00 00   'C.......'

This is SDO. Master (SDO client) sends read or write request for variable form Object Dictionary (CAN-ID = 0x600 + remote-node-id). SDO server responses (CAN-ID = 0x580 + node-id). SDO messages are harder to read, usually it is not necessary to know the details. For description see CiA301 or CO_SDO_state_t.
Line 21 is CO_SDO_ST_UPLOAD_INITIATE_REQ, index=0x1800, subindex=0x01
Line 22 is CO_SDO_ST_UPLOAD_INITIATE_RSP, same index and subindex, data=0x00000199
Equivalent CANopenNode gateway command: [1] 0x19 read 0x1800 1 x32.
line 25: (000.000710)  can0  619   [8]  40 00 18 03 00 00 00 00<EMAIL_ADDRESS>line 26: (000.002455)  can0  599   [8]  80 00 18 03 11 00 09 06   '........'

Line 25 is CO_SDO_ST_UPLOAD_INITIATE_REQ, index=0x1800, subindex=0x03
Line 26 is CO_SDO_ST_ABORT, same index and subindex, CO_SDO_abortCode_t = 0x06090011 #Sub index does not exist (this is strange, TPDO Inhibit time, Event timer and SYNC start value are missing)
This pattern then repeats and reads all TPDO and RPDO communication and mapping parameters. As in Python program.
line 183: (000.001001)  can0  619   [8]  23 00 18 01 99 01 00 80   '#.......'
line 184: (000.002439)  can0  599   [8]  60 00 18 01 00 00 00 00   '`.......'

Line 183 is CO_SDO_ST_DOWNLOAD_INITIATE_REQ, index=0x1800, subindex=0x01, data=0x80000199
Line 184 is CO_SDO_ST_DOWNLOAD_INITIATE_RSP, same index and subindex, OK
Equivalent CANopenNode gateway command: [1] 0x19 write 0x1800 1 x32 0x80000199 - disable PDO.
Then follows several similar commands configuring TPDOs and RPDOs.
After that NMT command sets node 0x19 to operational and then appears four different PDO messages on the bus.
Python is certainly most clear. More raw approach may also be with several CANopen gateway commands. Observing candump may be useful for understanding the details or for debugging.
I didn't decode all SDO commands and the rest of the program, but I hope I explained a little. It is important to know, where does TPDOs came from and what are the contents. See also some explanation of PDO re-mapping procedure.

Hello!
I wanted to check in on this issue, and see what the status was on potentially an example of this system transmitting and receiving pdos from another node!
Let me know how I can help as well would be happy to because this would be very useful!
Thank you!!

PDO tutorial is here now:
https://github.com/CANopenNode/CANopenDemo

His example is for CANopen for Python. This is another project.


I think that's something people would appreciate.

https://github.com/nostrnests/nests/issues/13#issue-2153832878

Duplicate


ghunt: add missing dependencies

Description of changes

Adds missing dependencies which I noticed when running ghunt email <email>.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 304315 run on x86_64-linux 1

2 packages built:

ghunt
ghunt.dist


ENAMETOOLONG error when searching in files

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: Latest
OS Version: Windows 10

Steps to Reproduce:

Search for something

Note that the error only occurs when I search for a string longer than 17 characters.

Likely due to having a long list of search exclude patterns, long workspace folder name, etc

/duplicate #98514

Still feels like an issue though, or at least a more accurate error message would be nice.
Are there any ways to debug this? I don't have any weird directory names, and no search patterns either as far as I know. Also, I've been working on this project without any problems for months and this occured randomly now.


What is wrong ?

Should be fixed now. Server had some logic issues.

close issue?

Yes. Can be closed.


Add support for automatic removal of old logs

This PR is in regard to #423. The code has been added and tested , but not yet integrated into glog.
Example usage:
vector<string> overdue_log_names = GetOverdueLogNames("/tmp", 1);

for (const auto& name : overdue_log_names) {
  unlink(name.c_str());
}

GetOverdueLogNames() will check all filenames under log_directory, and return a list of files whose last modified time is over the given days ( calculated with difftime() ), so that we can simply call unlink() on the filenames in the returned vector.
However, I'm not sure when is the right time to call it.

I signed it!

Guess the patch was implemented based on Linux. For Windows user, https://github.com/tronkko/dirent will be needed.

Guess the patch was implemented based on Linux. For Windows user, https://github.com/tronkko/dirent will be needed.

Thank you very much for your help! CI has passed on windows!
This is what I've done currently:
To enable log cleaner, in your project (log cleaner is disabled by default).
google::EnableLogCleaner(3);

And then your project will check if there are overdue logs whenever a flush is performed. In this example, the logs whose last modified times are >= 3 will be unlink(). It calls GetLoggingDirectories() to look for overdue logs.
You may also disable it at anytime.
google::DisableLogCleaner();

I try to check if a file is a glog file produced by the project it is linked in carefully. It will only remove a log if its filename matches: <program name>.<hostname>.<user name>.log.<severity level>, so logs from other projects should be untouched.
https://github.com/google/glog/blob/b8a61d70834977d18a524fc849ada2f6dea0d189/src/logging.cc#L871

@sergiud Sorry for asking but any chance this patch could be merged?
I've tested them on linux (4.19.66-gentoo) and windows 10, old log files can be removed automatically. This feature is disabled by default.

"It will only remove a log if its filename matches: ...log., so logs from other projects should be untouched."

@aesophor  If I use SetLogDestination API to change base_filename_, old logs will not delete automatically.  is a bug?

"It will only remove a log if its filename matches: ...log., so logs from other projects should be untouched."

@aesophor If I use SetLogDestination API to change base_filename_, old logs will not delete automatically. is a bug?

@cswuyg Yes, this is a bug. I'm working on a patch for this.

@cswuyg Fixed in #502


Fix "ghost" duplicates when sorting

Fixes issues #184 and #189 .
The key attribute should identify the row indifferently from its position.

Would love to get this merged so we can make this better for our users.

Please add this fix. It is already open for a month now.


feat: add support for multiple @key changes in same @model

Updated CLI to support multiple changes to @key directive in same @model during update. The CLI will
split the cloudformation stack into mulitple batches of update such that GSIs are added and removed
in separate pushes which out causing any cloudfomration deployment error. This change is behind
FeatureFlage graphqltransformer.enableIterativeGSIUpdates which is turned off by default.
The changes supported after enabling this GSI are

Rename of @key
Add and Remove multiple @keys at the same time in a single model
Add and remove fileds to @key

The change only helps in deployment of key changes and does not backfill the data if
the key is a composite key or remove column from Dynamo DB records when a key removed
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Overall looks good! My one high-level question / concern is around how we are storing the rollbacks. Currently it looks like each step has a rollback and a deployment "option" where the rollback option of the current step is the same as the deployment option of the previous step. This 1. creates a duplicate of all steps except the first and last and 2. creates a situation where we have to check for deploying or rolling back in many different places to pick the right option. Instead, I would propose each step only maintain the deployment "option" and then when executing the steps, push the current state onto a "rollback stack" before starting the next step. In the case of a failure, we can then swap over to executing the "rollback stack" until it is empty, thus reverting to the initial state.
Maybe there's something I'm not thinking of here, but it seems like this would allow us to use identical logic for deploying and rolling back

Your proposal works for how we depoy now but it would not work when we change the order in which we create and delete indexes. The current implementation makes the API miss indexes as we delete and create with no particular order. So if the API is using that index for list query, it wont be available after deleting the index. I am thinking of changing the order in which the indexes are created and deleted to keep the API working for as long as possible by during the push operations by using the following order

Create all the indexes by just deploying the API stack
Deploy the whole stack (so the API can use the new indexes. We also need to deploy the entire root stack to ensure if API has other dependecies like a function)
Delete the unused indexes

This work for all the cases except for case where an index is changing its hash or sort key. With this approach the rollback would not be as simple as what it is today. When you want to roll back after the deploying the rootstack in step 2, the rollback stack should be root stack from current cloud + previously deployed API stack.


Missing Type Annotations

There are missing type annotations for some functions in this library.
The typing module does not exist on CircuitPython devices so the import needs to be wrapped in try/except to catch the error for missing import. There is an example of how that is done here:
try:
    from typing import List, Tuple
except ImportError:
    pass

Once imported the typing annotations for the argument type(s), and return type(s) can be added to the function signature. Here is an example of a function that has had this done already:
def wrap_text_to_pixels(
    string: str, max_width: int, font=None, indent0: str = "", indent1: str = ""
) -> List[str]:

If you are new to Git or Github we have a guide about contributing to our projects here: https://learn.adafruit.com/contribute-to-circuitpython-with-git-and-github
There is also a guide that covers our CI utilities and how to run them locally to ensure they will pass in Github Actions here: https://learn.adafruit.com/creating-and-sharing-a-circuitpython-library/check-your-code In particular the pages: Sharing docs on ReadTheDocs and Check your code with pre-commit contain the tools to install and commands to run locally to run the checks.
If you are attempting to resolve this issue and need help, you can post a comment on this issue and tag both @foamyguy and @kattni or reach out to us on Discord: https://adafru.it/discord in the #circuitpython-dev channel.
The following locations are reported by mypy to be missing type annotations:

[ ] adafruit_displayio_layout/widgets/control.py:57
[ ] adafruit_displayio_layout/widgets/control.py:85
[ ] adafruit_displayio_layout/widgets/easing.py:79
[ ] adafruit_displayio_layout/widgets/easing.py:87
[ ] adafruit_displayio_layout/widgets/easing.py:95
[ ] adafruit_displayio_layout/widgets/easing.py:105
[ ] adafruit_displayio_layout/widgets/easing.py:115
[ ] adafruit_displayio_layout/widgets/easing.py:123
[ ] adafruit_displayio_layout/widgets/easing.py:134
[ ] adafruit_displayio_layout/widgets/easing.py:145
[ ] adafruit_displayio_layout/widgets/easing.py:153
[ ] adafruit_displayio_layout/widgets/easing.py:164
[ ] adafruit_displayio_layout/widgets/easing.py:175
[ ] adafruit_displayio_layout/widgets/easing.py:183
[ ] adafruit_displayio_layout/widgets/easing.py:194
[ ] adafruit_displayio_layout/widgets/easing.py:205
[ ] adafruit_displayio_layout/widgets/easing.py:213
[ ] adafruit_displayio_layout/widgets/easing.py:221
[ ] adafruit_displayio_layout/widgets/easing.py:229
[ ] adafruit_displayio_layout/widgets/easing.py:237
[ ] adafruit_displayio_layout/widgets/easing.py:247
[ ] adafruit_displayio_layout/widgets/easing.py:257
[ ] adafruit_displayio_layout/widgets/easing.py:267
[ ] adafruit_displayio_layout/widgets/easing.py:279
[ ] adafruit_displayio_layout/widgets/easing.py:291
[ ] adafruit_displayio_layout/widgets/easing.py:299
[ ] adafruit_displayio_layout/widgets/easing.py:309
[ ] adafruit_displayio_layout/widgets/easing.py:322
[ ] adafruit_displayio_layout/widgets/easing.py:330
[ ] adafruit_displayio_layout/widgets/easing.py:341
[ ] adafruit_displayio_layout/widgets/easing.py:352
[ ] adafruit_displayio_layout/widgets/easing.py:359
[ ] adafruit_displayio_layout/widgets/easing.py:372
[ ] adafruit_displayio_layout/widgets/widget.py:167
[ ] adafruit_displayio_layout/widgets/widget.py:198
[ ] adafruit_displayio_layout/widgets/widget.py:278
[ ] adafruit_displayio_layout/widgets/widget.py:295
[ ] adafruit_displayio_layout/layouts/grid_layout.py:51
[ ] adafruit_displayio_layout/layouts/grid_layout.py:290
[ ] adafruit_displayio_layout/layouts/grid_layout.py:308
[ ] adafruit_displayio_layout/widgets/switch_round.py:420
[ ] adafruit_displayio_layout/widgets/switch_round.py:664
[ ] adafruit_displayio_layout/widgets/switch_round.py:686
[ ] adafruit_displayio_layout/widgets/switch_round.py:783
[ ] adafruit_displayio_layout/widgets/switch_round.py:803
[ ] adafruit_displayio_layout/widgets/switch_round.py:827
[ ] adafruit_displayio_layout/widgets/switch_round.py:837
[ ] adafruit_displayio_layout/widgets/switch_round.py:849
[ ] adafruit_displayio_layout/widgets/switch_round.py:854
[ ] adafruit_displayio_layout/widgets/switch_round.py:887
[ ] adafruit_displayio_layout/widgets/switch_round.py:904
[ ] adafruit_displayio_layout/widgets/icon_widget.py:54
[ ] adafruit_displayio_layout/widgets/icon_widget.py:87
[ ] adafruit_displayio_layout/widgets/flip_input.py:84
[ ] adafruit_displayio_layout/widgets/flip_input.py:329
[ ] adafruit_displayio_layout/widgets/flip_input.py:431
[ ] adafruit_displayio_layout/widgets/flip_input.py:447
[ ] adafruit_displayio_layout/widgets/flip_input.py:506
[ ] adafruit_displayio_layout/widgets/flip_input.py:525
[ ] adafruit_displayio_layout/widgets/flip_input.py:580
[ ] adafruit_displayio_layout/widgets/flip_input.py:622
[ ] adafruit_displayio_layout/widgets/icon_animated.py:77
[ ] adafruit_displayio_layout/widgets/icon_animated.py:131
[ ] adafruit_displayio_layout/widgets/icon_animated.py:168
[ ] adafruit_displayio_layout/widgets/icon_animated.py:263

Picking this one up for the PyCon 2023 sprint.

@FoamyGuy Hi, working on this, but I'm a bit stumped. in switch_round.py , selected and contains methods are calling the superclass, with a touchpoint argument. According to the docs in the comments, touchpoint should be of type Tuple[float, float], but the calls here include a third parameter, which is not expected by the superclass:
https://github.com/adafruit/Adafruit_CircuitPython_DisplayIO_Layout/blob/main/adafruit_displayio_layout/widgets/switch_round.py#L810
Why the extra parameter? Am I missing something? Thanks for helping out.

@cguardia Hiya, the extra parameter is the pressure that is needed for the touchscreen.
https://github.com/adafruit/Adafruit_CircuitPython_Touchscreen/blob/d392b3135ced990cda052536b8451a37238fb540/adafruit_touchscreen.py#L128

@jposada202020 Thanks a lot. So I guess that means I should change all doc strings  that refer to this as just an x,y tuple, right?

@cguardia I think is a good idea. We could document, afterwards, if the pressure parameter is exposed to the user. A question, you mentioned a tuple of floats, I think they are ints. But I might be wrong.

touchpoints should indeed contain int values. And agree with the above, we can update the documentation to mention the optional pressure argument (I don't think all screen types have this capability so some will not have that 3rd value)
Perhaps:
Tuple[int, int, Optional[int]]

@FoamyGuy Thanks for the guidance. One more question: calls that use touch_point as an argument usually get List[int] in the call, rather than the tuple. Should we allow lists as well on the type description, or change calls that pass in lists to pass in tuples instead?

@cguardia I've had a poke around and I think we want to allow tuples or lists, at least for now. the Touchscreen library uses tuples, but this other capacitive touch screen library says it returns lists: https://docs.circuitpython.org/projects/focaltouch/en/latest/api.html#adafruit_focaltouch.Adafruit_FocalTouch.touches


Calculated maxsteps not used in training loop

jbross created the following on Jun 09:
Initially the maxsteps is calculated as minimum between user provided config.maxsteps and maxsteps derived from user provided config.epochs in line 260:
maxsteps = min(config.maxsteps, math.ceil((config.epochs * len(reader)) / (config.bsize * config.nranks)))

However, in the actual loop over the training batches in line 383 it uses the config.maxsteps instead of the calculated maxsteps value:
for batch_idx, BatchSteps in zip(range(start_batch_idx, config.maxsteps), reader):

This does not seem to be intended, as the config.epochs has no effect at all.
Original issue: https://github.ibm.com/ai-foundation/PrimeQA/issues/228

@franzmpub : I think you had looked into this already. Can we close it ? Please update the status/pipeline of the issue accordingly.


TSN features

How to build TSN features for our own datasets

To extract the RGB and Flow features, you can use this repository TSN-yjxiong.
And also, you can use mmaction2 and follow the instruction to extract the TSN feature.
Before that, you must extract the optical flow frame through denseflow.


Problems with generated QClasses since 4.1.3

4.1.3 introduced some "PropertyHandler/Config" and this changed the behavior how fields/properties are being generated.
There is an abstract "base entity" which defines the getter for the ID field which is declared in the subclass:
@MappedSuperclass
public abstract class BaseEntity implements Serializable {
    public abstract Long getId();
[...]

This field is no longer created in the generated class:
/**
 * QBaseEntity is a Querydsl query type for BaseEntity
 */
@Generated("com.querydsl.codegen.SupertypeSerializer")
public class QBaseEntity extends EntityPathBase<BaseEntity> {
[...]

It seems thats not possible with the maven-plugin to set the configuration of the PropertyHandling to another value than the default PropertyHandling.JPA.
I would expect that configuration option PropertyHandling.ALL would restore the former behavior.

To get rid of the old issues, i've retested it with the current 4.2.1.
And i had no more problems so far, so i close this issue by myself to clear the view for the important issues.

Ahh, thanks so much.


New folder - naming validation is case insensitive

VSCode Version: 1.16.1
OS Version: OS X 10.10.5

Steps to Reproduce:

Reproduces without extensions: Yes

It's because of the file system of OS X...


GH-33619:  [Documentation] Update PR template

Rationale for this change
I will keep this as a draft for now please feel free to suggest additional changes!
What changes are included in this PR?
I have switched to smaller headers for now and removed the first head + closes keyword to avoid redundancy.

Closes: #33619

Looks good to me! Thanks for making this change, @assignUser!

Following up on the discussion about stripping comments from the pull request description when merging, I believe adding the following around line 583 of merge_arrow_pr.py should work:

Short term that would be good to do to avoid those long noisy commit messages. Longer term, IMO we should discuss using only the title of the PR.

Whoops, sorry, the linter was failing on the code line in archery .. This will be fixed by https://github.com/apache/arrow/pull/15067


Reolink cameras

Received some Reolink cameras from manufacturer to record videos about them.
Since I'm not going for some deep analysis of camera per se, I'll be focusing on various types of integrations that will include:

Synology DSM/Surveillance Station
HACS Reolink custom component
ONVIF
ffmjpeg
Also some possible automations that could be fun to integrate and useful in real-life.

I've finally redone cameras. Old outdoor camera, that was used for Facebox face recognition is now removed and it will be used to monitor 3D printer (when I find good place to install it). New Reolink camera (410) has been placed outside and is now used by Facebox.
Also, while I was working on it, I removed my only useful NodeRed flow I had - to trigger face recognition when Synology Surveillence station detects motion. It now directly triggers Home Assistant webhook and that should also shorten and speedup the process too.


Webpack UglifyJs is failing each time i run a production build

ERROR in app.js from UglifyJs Unexpected token: name (e) [./node_modules/vue-google-login/dist/vue-google-login.min.js:1,2044][app.js:58921,2001] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Hi, I tried to replicate the issue but I couldn't manage to do it.
I've updated the package to add some options and at the same time updated the tool I use to bundle the component.
Update it and try again.
You'll need to make some small adjustment since now accepts an object and not a string.
Let me know if it keeps happening.

@rmartide Thank you! I will make the adjustments and let you know how it goes!

@rmartide Sorry for the delay. I happened to pull your latest changes and did not help either but however got some hits on how to get past this https://github.com/JeffreyWay/laravel-mix/issues/790 and it helped


Remove support for ={...} substitutions or make it optional

I'm not sure why this was added because it's neither actual HAML or Django. It breaks existing templates where users have used things like {href: "foo={{ bar }}"}.
I'd like to either remove it for a version 1.0 release, or at least make a way to disable it.

I'm not fully at ease with breaking backward compatibility but the fact that this is breaking valid django syntax is a problem. I think that it's there to provide an alternative to ruby's #{var}. This is apparently there to remove another older syntax https://github.com/jessemiller/HamlPy/blob/acb79e14381ce46e6d1cb64e7cb154751ae02dfe/hamlpy/elements.py#L135
It would be great to see if this syntax is used somewhere but github search seems broken for that:
https://github.com/search?utf8=✓&q="%3D{"+extension%3Ahaml+extension%3Ahamlpy&type=Code&ref=advsearch&l=&l=

Breaking backward compatibility is certainly to be avoided where possible, but I think if we made a 1.0 major release then it could be justified on the basis that:

It's not HAML and doesn't provide any advantages over the actual HAML syntax of #{...}
It can break templates using Django {{...}} tags, which users like us have been using with the old 0.82 release of hamlpy.

For the sake of users who have used ={...} we could make support configurable - but I do feel that even then the default should be disabled, and the documentation should warn users against using both {{...}} and ={...} in the same templates.

@Psycojoker Github search doesn't support symbol characters apparently. We need to move forward sooner than later on this, and as I see it there are 2 options:

Remove support for ={ (and any other non-HAML oddities?) and make a 1.0 release. Breaking backward compatibility isn't ideal but having a major version release sets appropriate user expectations.
Make support for ={ configurable via a setting

I prefer 1 but am ok with 2 if that's your desire.

Had a go at option 2 (see https://github.com/Psycojoker/django-hamlpy/compare/master...Psycojoker:optional_sub_syntax?expand=1) tho still feel removing the syntax would be preferable. Your call. What you think?

Why would you be against option 2 here? You branch seems pretty good (and, while I prefere simplicity, maybe the default option mecanism + option for rendering could be useful for the futur)
I would prefer not to break backward compatibility but that's not an ultimate no-go for me in this case.

I think we have much better chance of adoption and continued use if we are as close to actual HAML as possible. So this is my proposal:

We make it optional
We remove it from the reference documentation
At some point in future we make a 1.0 release which drops support for it

I can turn the branch above into a PR that does 1 and 2.

Works for me, just add "display a deprecation warning" between 2 and 3.

Added in #24. Default is that support is continues to be enabled, so no deprecation needed just yet.


"No rule to make target" error when building

I am completely new to Linux. I have installed a fresh Ubuntu 16.04 version and I have tried to build the Director. However I am getting this error:
make[6]: *** No rule to make target '/usr/lib/x86_64-linux-gnu/libSM.so', needed by 'lib/libvtkDRCFilters.so'.  Stop.
make[6]: Leaving directory '/home/gian/git/director/build/src/director-build'
CMakeFiles/Makefile2:1123: recipe for target 'src/vtk/DRCFilters/CMakeFiles/vtkDRCFilters.dir/all' failed
make[5]: *** [src/vtk/DRCFilters/CMakeFiles/vtkDRCFilters.dir/all] Error 2
make[5]: Leaving directory '/home/gian/git/director/build/src/director-build'
Makefile:138: recipe for target 'all' failed
make[4]: *** [all] Error 2
make[4]: Leaving directory '/home/gian/git/director/build/src/director-build'
CMakeFiles/director.dir/build.make:116: recipe for target 'src/director-stamp/director-build' failed
make[3]: *** [src/director-stamp/director-build] Error 2
make[3]: Leaving directory '/home/gian/git/director/build'
CMakeFiles/Makefile2:71: recipe for target 'CMakeFiles/director.dir/all' failed
make[2]: *** [CMakeFiles/director.dir/all] Error 2
make[2]: Leaving directory '/home/gian/git/director/build'
Makefile:83: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/home/gian/git/director/build'
Makefile:22: recipe for target 'superbuild' failed
make: *** [superbuild] Error 2

I noticed there is no libSM.so in /usr/lib/x86_64-linux-gnu/, but I found a libSM.so.6 and libSM.so.6.0.1. Any idea on how to solve this? Thanks

Thanks for posting.  I recently updated the list of required dependencies and I think I am missing some items.  Please try installing this set:
sudo apt-get install build-essential cmake libqt4-dev python-dev python-numpy libgl-dev libglu-dev libxext-dev libxt-dev
If you'd like, you can download precompiled binaries of Director so that you don't have to compile from source:
https://bintray.com/patmarion/director/director/0.1.0-153-ge0a5a7b#files

@patmarion thanks for replying. I managed to solve in this way:
sudo rm -f /usr/lib/x86_64-linux-gnu/libSM.so
sudo ln -s /usr/lib/x86_64-linux-gnu/libSM.so.6 /usr/lib/x86_64-linux-gnu/libSM.so
However, I am very new to Linux so I have one more question. After building do I have to install it? Because if I type make install inside director/build nothing happens.

Ok, glad you got it working.  But I would not recommend the solution you posted here, you should never sudo rm anything from /usr/lib without a very good reason.  I suspect that you were just missing the development version of the library which may have been installed by one of the dependencies I listed above.


[stable/nginx-ingress] No empty string clusterip

Signed-off-by: Naseem<EMAIL_ADDRESS>
Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.

What this PR does / why we need it:
Please see #19041 and https://github.com/helm/helm/issues/6378#issuecomment-557746499
helm3 introduces 3 way merge, i am stuck with a release where omitClusterIP was not set and is now managed by helm3. This would probably help
Special notes for your reviewer:
We may be able to remove the omitclusterIP workaround if we wish, provided nobody has hardcoded clusterIP: "" in their values files
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

cc @taharah @ChiefAlexander

/ok-to-test
/lgtm

/retest

/test tide


您好，我想尝试自己打包生成一个安装包，为什么软件时间日期显示错误，显示1980年

请教下，我该怎么做 能让日期显示正常。

感谢您创建此问题！
然而，由于您没有使用英语，我们很难进行处理。如果可能，请您将此问题描述修改为英文。请您直接编辑此问题，不要提交回复或新问题。
请注意，机器翻译服务通常无法处理用于讨论软件问题所需的技术性语言，所以我们推荐您寻找一位熟悉相关技术的人来协助翻译。
如果您无法做到，没有关系。此问题已经被标记为需要翻译，等待社区成员志愿进行翻译。

Thanks for creating this issue!
Unfortunately, as the description is not in English, it's hard for us to work on. If possible, please edit the original issue to be in English.
In our experience, automatic translation services are not able to handle the technical language needed to effectively communicate about software issues, so we recommend finding a human translator who is familiar with our technology to help.
Otherwise, we have marked this as needing translation and a community member may be able to help out.


Failure to build - cannot find <ncurses.h>

I've run RUST_BACKTRACE=1 cargo install --path . to install, but get the following error:
  /home/ghost/Projects/tap/target/release/build/ncurses-655630ba6bec2a2c/out/chtype_size.c:6:10: fatal error: ncurses.h: No such file or directory
      6 | #include <ncurses.h>
        |          ^~~~~~~~~~~
  compilation terminated.
  thread 'main' panicked at 'assertion failed: command.status().expect(\"compilation failed\").success()', /home/ghost/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ncurses-5.101.0/build.rs:105:5
  stack backtrace:
     0: rust_begin_unwind
     1: core::panicking::panic_fmt
     2: core::panicking::panic
     3: build_script_build::check_chtype_size
     4: build_script_build::main
     5: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
error: failed to compile `tap v0.4.7 (/home/ghost/Projects/tap)`, intermediate artifacts can be found at `/home/ghost/Projects/tap/target`.

I've installed the ncurses package (ncurses-6.4_2), but this didn't help.

OS: Void Linux

NB: I have no idea how rust works, I'm just following the error message, so feel free to close the issue if I'm just missing something simple.

Hi, thanks for raising this issue.
It looks like the ncurses header file either doesn't exist or it's not linked to. Can you try installing the ncurses-devel package, then running cargo clean from within the project directory. This might fix this issue

This works! Thanks.


Add Sort Changes By Extension in the CSM View&Sort

Feature request
Add 'Sort Changes By Extension' in the CSM View&Sort

wtf guys?

Thank you very much for your contribution. Please note that the sort key is also present as a setting so if we were to add a new sort key, we would also have to update the scm.defaultViewSortKey setting with the new sort key.
I was also wondering if you could illustrate a scenario in which you would leverage sorting by extension. Thanks!

Thank you very much for your contribution. Please note that the sort key is also present as a setting so if we were to add a new sort key, we would also have to update the scm.defaultViewSortKey setting with the new sort key.
I was also wondering if you could illustrate a scenario in which you would leverage sorting by extension. Thanks!

What if I have >2K changes and want to upload just a bunch of .php


MariaDB crashing in submit13 after stop/start

Impact of the bug
MariaDB in submit13
Describe the bug
As it has been reported during the past days and today in the WMCore meeting, the FNAL agents were having issues and slowly progressing through the workload.
Based on that and some reports of "lock wait timeout" in a few components, I decided to:

stop wmagent components
stop wmagent container
stop mariadb container (the docker stop command actually timed out, hence the container was forcebly killed)
started mariadb container (which failed to connect to localhost)
and everything else went downhill!

After a few attempts, MariaDB managed to be started from inside the container and I can access the database prompt and run queries.
However, after starting WMAgent, I see that every single component crashed while communicating with the RDMBS.
How to reproduce it
Not sure
Expected behavior
The database to be fully operational and keeping up with the load that we have always executed.
Additional context and error message
Full error log path is /data/srv/mariadb/current/logs/error.log.
Inside this log, we can see the moment that the service was stopped and its logs (everything looks normal to me):
2024-10-07 17:50:36 0 [Note] /usr/sbin/mariadbd (initiated by: xxx[xxx] @ localhost []): Normal shutdown
2024-10-07 17:50:36 0 [Note] InnoDB: FTS optimize thread exiting.
2024-10-07 17:50:36 0 [Note] InnoDB: Starting shutdown...
2024-10-07 17:50:36 0 [Note] InnoDB: Dumping buffer pool(s) to /data/srv/mariadb/10.6.5/install/database/ib_buffer_pool
2024-10-07 17:50:36 0 [Note] InnoDB: Restricted to 129552 pages due to innodb_buf_pool_dump_pct=25
2024-10-07 17:50:36 0 [Note] InnoDB: Buffer pool(s) dump completed at 241007 17:50:36
241007 17:52:26 mysqld_safe Starting mariadbd daemon with databases from /data/srv/mariadb/10.6.5/install/database

Followed by a restart of the mariadb container:
241007 17:52:26 mysqld_safe Starting mariadbd daemon with databases from /data/srv/mariadb/10.6.5/install/database
2024-10-07 17:52:27 0 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used;  This may cause repliction to break when this server acts as a master and has its hostname changed! Please use '--log-basename=cmsgwms-submit13.fnal.gov' or '--log-bin=mariadbd-bin' to avoid this problem.
2024-10-07 17:52:27 0 [Note] /usr/sbin/mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal-log) starting as process 346 ...
2024-10-07 17:52:27 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2024-10-07 17:52:27 0 [Note] InnoDB: Number of pools: 1
2024-10-07 17:52:27 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-10-07 17:52:27 0 [Note] InnoDB: Using Linux native AIO
2024-10-07 17:52:27 0 [Note] InnoDB: Initializing buffer pool, total size =<PHONE_NUMBER>, chunk size = 134217728
2024-10-07 17:52:27 0 [Note] InnoDB: Completed initialization of buffer pool
2024-10-07 17:52:27 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=290549958163,290550247415
2024-10-07 17:52:38 0 [Note] InnoDB: Starting final batch to recover 82737 pages from redo log.
2024-10-07 17:52:42 0 [Note] InnoDB: To recover: 82385 pages from log
2024-10-07 17:52:57 0 [Note] InnoDB: To recover: 81951 pages from log
2024-10-07 17:53:12 0 [Note] InnoDB: To recover: 81533 pages from log
2024-10-07 17:53:27 0 [Note] InnoDB: To recover: 81378 pages from log
241007 17:53:35 mysqld_safe Starting mariadbd daemon with databases from /data/srv/mariadb/10.6.5/install/database
2024-10-07 17:53:35 0 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used;  This may cause repliction to break when this server acts as a master and has its hostname changed! Please use '--log-basename=cmsgwms-submit13.fnal.gov' or '--log-bin=mariadbd-bin' to avoid this problem.
2024-10-07 17:53:35 0 [Note] /usr/sbin/mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal-log) starting as process 708 ...
2024-10-07 17:53:35 0 [ERROR] mariadbd: Can't lock aria control file '/data/srv/mariadb/10.6.5/install/database/aria_log_control' for exclusive use, error: 11. Will retry for 30 seconds
2024-10-07 17:53:42 0 [Note] InnoDB: To recover: 64520 pages from log
2024-10-07 17:53:57 0 [Note] InnoDB: To recover: 54171 pages from log
2024-10-07 17:54:05 0 [ERROR] mariadbd: Got error 'Could not get an exclusive lock; file is probably in use by another process' when trying to use aria control file '/data/srv/mariadb/10.6.5/install/database/aria_log_control'
2024-10-07 17:54:05 0 [ERROR] Plugin 'Aria' init function returned error.
2024-10-07 17:54:05 0 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
2024-10-07 17:54:05 0 [Note] InnoDB: Compressed tables use zlib 1.2.11

and while I was trying to stop/start the service (without stopping and looking into the logs to see what was going on), we see this in the same log:
2024-10-07 17:57:13 0 [Note] InnoDB: Starting final batch to recover 82737 pages from redo log.
2024-10-07 17:57:25 0 [Note] InnoDB: To recover: 81785 pages from log
2024-10-07 17:57:27 0 [Note] InnoDB: To recover: 82402 pages from log
2024-10-07 17:57:40 0 [Note] InnoDB: To recover: 81512 pages from log
2024-10-07 17:57:42 0 [Note] InnoDB: To recover: 82102 pages from log
2024-10-07 17:57:55 0 [Note] InnoDB: To recover: 81373 pages from log
2024-10-07 17:57:57 0 [Note] InnoDB: To recover: 81676 pages from log
2024-10-07 17:58:00 0 [ERROR] InnoDB: Page [page id: space=0, page number=7197] log sequence number<PHONE_NUMBER>54 is in the future! Current system log sequence number<PHONE_NUMBER>11.
2024-10-07 17:58:00 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.

I've been trying to get a dump of the database, but it keeps crashing at the same table:
(MariaDB-10.6.5) [xxx@submit13:current]$ mariadb-dump --socket=/var/run/mysqld/mariadb.sock wmagent --single-transaction --verbose > /data/admin/wmagent/mariadb_bkp2/backup-file-2.sql
...
-- Retrieving table structure for table wmbs_sub_files_acquired...
-- Sending SELECT query...
-- Retrieving rows...
mariadb-dump: Error 2013: Lost connection to server during query when dumping table `wmbs_sub_files_acquired` at row: 244986

this has been attempted in normal mode and recovery mode 1, such as:
2024-10-08 10:42:55 0 [Note] InnoDB: !!! innodb_force_recovery is set to 1 !!!

after updating the startup command in the manage script.
According to this documentation:
https://mariadb.com/kb/en/innodb-recovery-modes/
I could try recovery modes up to 3. After that, the recovery is more "aggressive"...

And MariaDB databases check process didn't manage to complete, as the service crashed after scanning databases/tables for an hour or so:
(MariaDB-10.6.5) [xxx@submit13:current]$ mariadb-check --verbose --socket=/var/run/mysqld/mariadb.sock --all-databases
Processing databases
information_schema
mysql
mysql.column_stats                                 OK
...
mysql.transaction_registry                         OK
performance_schema
sys
sys.sys_config                                     OK
wmagent
wmagent.bl_runjob                                  OK
...
wmagent.wmbs_frl_workunit_assoc                    OK
mariadb-check: Got error: 2013: Lost connection to server during query when executing 'CHECK TABLE ... '

The error log says:
2024-10-08 14:46:21 6 [ERROR] InnoDB: In pages [page id: space=27, page number=294730] and [page id: space=27, page number=23462] of index `wmbs_job_unique` of table `wmagent`.`wmbs_job`, index tree level 1
InnoDB: broken FIL_PAGE_NEXT or FIL_PAGE_PREV links
2024-10-08 14:46:21 6 [ERROR] InnoDB: In page 23462 of index `wmbs_job_unique` of table `wmagent`.`wmbs_job`, index tree level 1
2024-10-08 14:46:21 6 [ERROR] InnoDB: Node ptrs differ on levels > 0
InnoDB: node ptr PHYSICAL RECORD: n_fields 5; compact format; info bits 0
 0: len 30; hex<PHONE_NUMBER>3565312d626337302d346333352d393564352d303331343462; asc abfb55e1-bc70-4c35-95d5-03144b; (total 39 bytes);
 1: SQL NULL;
 2: SQL NULL;
 3: len 4; hex 8064bab4; asc  d  ;;
 4: len 4; hex 00005ba6; asc   [ ;;
InnoDB: first rec PHYSICAL RECORD: n_fields 5; compact format; info bits 0
 0: len 30; hex<PHONE_NUMBER>6163392d326633392d343165322d623935622d623361663861; asc ac10bac9-2f39-41e2-b95b-b3af8a; (total 36 bytes);
 1: len 30; hex 2f646174612f7372762f776d6167656e742f63757272656e742f696e7374; asc /data/srv/wmagent/current/inst; (total 266 bytes);
 2: len 30; hex 2f646174612f7372762f776d6167656e742f63757272656e742f696e7374; asc /data/srv/wmagent/current/inst; (total 279 bytes);
 3: len 4; hex 80252fb3; asc  %/ ;;
 4: len 4; hex 00065291; asc   R ;;

2024-10-08 14:52:07 6 [ERROR] InnoDB: Corruption of an index tree: table `wmagent`.`wmbs_job` index `wmbs_job_unique`, father ptr page no 50211, child page no 50222
PHYSICAL RECORD: n_fields 4; compact format; info bits 0
 0: len 30; hex<PHONE_NUMBER>3766372d663533652d346330312d613863622d653662633134; asc 48bd57f7-f53e-4c01-a8cb-e6bc14; (total 40 bytes);
 1: len 30; hex 2f646174612f7372762f776d6167656e742f63757272656e742f696e7374; asc /data/srv/wmagent/current/inst; (total 193 bytes);
 2: len 30; hex 2f646174612f7372762f776d6167656e742f63757272656e742f696e7374; asc /data/srv/wmagent/current/inst; (total 206 bytes);
 3: len 4; hex 8086efb2; asc     ;;
2024-10-08 14:52:07 6 [Note] InnoDB: n_owned: 0; heap_no: 2; next rec: 585
PHYSICAL RECORD: n_fields 5; compact format; info bits 0
 0: len 30; hex<PHONE_NUMBER>3766372d663533652d346330312d613863622d653662633134; asc 48bd57f7-f53e-4c01-a8cb-e6bc14; (total 40 bytes);
 1: len 30; hex 2f646174612f7372762f776d6167656e742f63757272656e742f696e7374; asc /data/srv/wmagent/current/inst; (total 193 bytes);
 2: len 30; hex 2f646174612f7372762f776d6167656e742f63757272656e742f696e7374; asc /data/srv/wmagent/current/inst; (total 206 bytes);
 3: len 4; hex 8086efc1; asc     ;;
 4: len 4; hex 0000c423; asc    #;;
2024-10-08 14:52:07 6 [Note] InnoDB: n_owned: 0; heap_no: 40; next rec: 1181
2024-10-08 14:52:07 6 [ERROR] [FATAL] InnoDB: You should dump + drop + reimport the table to fix the corruption. If the crash happens at database startup. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery. Then dump + drop + reimport.
241008 14:52:07 [ERROR] mysqld got signal 6 ;

I collected a dump of the wmagent database and all the files are available under:
[cmsdataops@cmsgwms-submit13 wmagent]$ ls -lth /data/dockerMount/admin/wmagent/mariadb_table_bkp/
total 19G
-rw-r--r-- 1 xxx xxx  15G Oct  8 19:01 backup-file.sql
-rw-r--r-- 1 xxx xxx 3.1G Oct  8 15:16 wmbs_job.sql
-rw-r--r-- 1 xxx xxx 2.3K Oct  8 15:03 wmbs_workunit.sql
-rw-r--r-- 1 xxx xxx 1.4M Oct  8 15:03 wmbs_workflow_output.sql
-rw-r--r-- 1 xxx xxx  14M Oct  8 15:03 wmbs_workflow.sql
-rw-r--r-- 1 xxx xxx 2.9K Oct  8 15:03 wmbs_users.sql
-rw-r--r-- 1 xxx xxx 3.7M Oct  8 15:03 wmbs_subscription_validation.sql
-rw-r--r-- 1 xxx xxx 1.9M Oct  8 15:02 wmbs_subscription.sql
-rw-r--r-- 1 xxx xxx 2.2K Oct  8 15:02 wmbs_sub_types.sql
-rw-r--r-- 1 xxx xxx  11M Oct  8 15:02 wmbs_sub_files_failed.sql
-rw-r--r-- 1 xxx xxx 470M Oct  8 15:00 wmbs_sub_files_complete.sql
-rw-r--r-- 1 xxx xxx  19M Oct  8 14:50 wmbs_sub_files_available.sql
-rw-r--r-- 1 xxx xxx 4.3M Oct  8 14:10 wmbs_sub_files_acquired.sql

where backup-file.sql contains most of the tables, but given that it was crashing at the table wmbs_sub_files_acquired, I decided to dump table by table for the remaining of the database.
Now I am going to try to load these dumps in a fresh new instnce (testing in submit14), if everything goes well, then I will perform the same.
It is important to mention though that the following 2 tables are crashing MariaDB:
mariadb-dump: Error 2013: Lost connection to server during query when dumping table `wmbs_sub_files_acquired` at row: 244986
mariadb-dump: Error 2013: Lost connection to server during query when dumping table `wmbs_sub_files_available` at row: 1095366

so their dump is likely incomplete, but I fear we will have to move forward with it like this.

Other than the 2 tables reported above - listing here as well wmbs_sub_files_acquired and wmbs_sub_files_available - which were reported with corrupted data causing MariaDB table dump to be only partial, I believe the remaining of the wmagent tables have been fully dumped (backed-up).
After that, I had to drop the wmagent database in submit13, which led to failures initializing a new database schema. With that, my next step was a manage mysql-clean and the full removal of /data/dockerMount/srv/mariadb.
Next, I redeployed MariaDB, then restarted the WMAgent container, such that it could create the database schema.
Next, I started loading the dump file located at:
[xxx@submit13 ~]$ ls /data/dockerMount/srv/mariadb/current/logs/backup-file.sql

into MariaDB (which is no longer running in recovery mode).
Loading this data took about 2h in total, before starting things up, I decided to run a database check and this time everything is reported OK and no crashes have been seen:
(MariaDB-10.6.5) [xxx@submit13:logs]$ mariadb-check --socket=/var/run/mysqld/mariadb.sock --all-databases
mysql.column_stats                                 OK
...
wmagent.wmbs_workflow_output                       OK
wmagent.wmbs_workunit                              OK
(MariaDB-10.6.5) [xxx@submit13:logs]$

In general, I was following a few of the MariaDB materials and explanations of comments. For the dump/load procedure, this link summarizes this well: https://mariadb.com/kb/en/innodb-recovery-modes/#fixing-things
NOW it is time to try to restart this agent and see how things evolve.

The agent did start up, there are some components complaining about localhost couchdb timing out, but things are flowing.
I somehow also expect other failures, as the database is definitely missing either sub_available or sub_acquired files, so there might be some jobs (and workflows) with an "unexplainable" outcome.

The agent is almost drained now, but it required some workflow manipulation, which the P&R team promptly and kindly helped us out. Here is a summary of a few of those:
A few workflows were hitting an unique constraint in WorkQueueManager, e.g.:
Error: (MySQLdb.IntegrityError) (1062, "Duplicate entry 'cmsunified_task_TOP-RunIISummer20UL18wmLHEGEN-00710__v1_T_240...' for key 'uniq_wf_name'")

and the following workflows have been either aborted or force-completed - such that the component can move forward:
cmsunified_task_TOP-RunIISummer20UL18wmLHEGEN-00710__v1_T_240212_095605_5779
haozturk_task_BPH-GenericGSmearS-00004__v1_T_231219_141016_1739
cmsunified_task_TOP-RunIISummer20UL16wmLHEGENAPV-00739__v1_T_240810_102700_6505
ACDCs with no work left in the acdcserver (probably irrelevant, but just logging it here as well):
cmsunified_ACDC2_task_EXO-Run3Summer23wmLHEGS-00828__v1_T_240805_004657_9100
cmsunified_ACDC2_task_GEN-Run3Summer23BPixwmLHEGS-00503__v1_T_240805_002817_3223
Once this agent is fully drained, I think we can close this issue out.

 With that, I think we can close this issue out.


v3.28.0 crash on cloudflare_account update

Confirmation

[X] My issue isn't already found on the issue tracker.
[X] I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version
Terraform v0.13.7
I know this is pretty old but the environement I'm working with cannot be upgraded at the moment
Affected resource(s)
cloudflare_account
Terraform configuration files
terraform {
  required_providers {
    cloudflare = {
      source = "cloudflare/cloudflare"
      version = "3.28.0"
    }
  }
}

provider "cloudflare" {
  account_id = "REDACTED"
  api_token = "REDACTED"
}

resource "cloudflare_account" "test" {
  name = "test-new"
  enforce_twofactor = true
  type = "standard"
}

Link to debug output
Debug output has a lot of sensitive information, can provide if absolutely needed
Panic output
See panic at bottom of https://gist.github.com/jfchevrette/a6fe4852b3e58a248716469891b4d8ac
Expected output
# cloudflare_account.test will be updated in-place
  ~ resource "cloudflare_account" "test" {
      ~ enforce_twofactor = false -> true
        id                = "2a98f16f04b782a94f252de9a6277653"
      ~ name              = "test" -> "test-new"
        type              = "standard"
    }

Actual output
See https://gist.github.com/jfchevrette/a6fe4852b3e58a248716469891b4d8ac
Steps to reproduce
See above config and expected output
Additional factoids
No response
References
No response

I've updated the original message after reproducing the issue using a test account
Provider the full TF_LOG=DEBUG output here https://gist.github.com/jfchevrette/a6fe4852b3e58a248716469891b4d8ac


Dialog - Add Theme and ActionHeader

Summary
Dialog has been updated to consume ActionHeader. I have also added theme variables to Dialog, Modal, and ModalOverlay.
Deployment URL
https://terra-core-deployed-pr-1209.herokuapp.com/static/#/site/dialog
I also updated Dialog to use subcomponents for body and footer. Since I am having to pass the header/footer/body content to ContentContainer via props, I was unsure if how I have it currently working is the best approach. If not, I welcome tips on a better path.
EDIT: Subcomponent implementation has been removed for simplicity.
@cerner/terra

Functional verification completed. The dialog examples display and function correctly with the action header changes applied.

Tested: PR #1209
Resolves: Issue #1157 #416

Reviewed:

Follows pattern considerations documented in Cerner Design Standards: Dialogs
Follows UX defined design implementation for Terra-Dialog
Follows defined theme designs: terra-theme

Future Enhancments:

Action header will be uplifted to universal design in the future
Action footer will be created with universal design in the future

Verified by @scottwilmarth and @neilpfeiffer, Ready for Release after color updates.

Pass

UX Review


Ft/chain guards

Codecov Report

Merging #26 (ce076a3) into main (ce8eab6) will increase coverage by 0.32%.
The diff coverage is 91.17%.

@@             Coverage Diff              @@
##               main      #26      +/-   ##
============================================
+ Coverage     70.49%   70.82%   +0.32%
  Complexity      689      689
============================================
  Files            35       35
  Lines          2688     2711      +23
  Branches        290      291       +1
============================================
+ Hits           1895     1920      +25
+ Misses          641      639       -2
  Partials        152      152

Impacted Files
Coverage Δ

...in/java/io/cucumber/core/plugin/JsonFormatter.java
58.33% <0.00%> (ø)

.../main/java/com/decathlon/tzatziki/utils/Guard.java
93.24% <92.53%> (+6.96%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce8eab6...ce076a3. Read the comment docs.


WIP: Prepare 2.0 release

Update README files.
Update Travis config.
Update dependencies.

Yeah, I need to test Expo project, prepare migration file and check latest XCode version.

@dariuszseweryn It's no longer WIP :)


SVM optimization with GPGO: objective function not improving after several iterations and different config of acq funct and surrogate model

`from pyGPGO.covfunc import squaredExponential
from pyGPGO.surrogates.GaussianProcess import GaussianProcess
from pyGPGO.surrogates.RandomForest import RandomForest
from pyGPGO.GPGO import GPGO
from pyGPGO.acquisition import Acquisition
from sklearn import datasets
from sklearn import svm
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_wine
import collections
df=load_wine()
collections.Counter(df['target'])
xtrain, xtest, ytrain, ytest = train_test_split(df['data'],df['target'],test_size=0.24,random_state=seed)
def accuracy_SVC(C,coef0,gamma):
clf = svm.SVC(C=C,coef0=coef0,gamma=gamma,kernel='sigmoid')
scores = cross_val_score(clf, xtrain, ytrain, cv=10)
return (scores.mean())
from pyGPGO.covfunc import matern32
from pyGPGO.covfunc import gammaExponential
covf=gammaExponential()
covf2=matern32()
GP = GaussianProcess(covf)
RF = RandomForest()
acq = Acquisition(mode='ProbabilityImprovement')
acq3= Acquisition(mode='UCB')
import numpy as np
bo = GPGO(GP,acq,accuracy_SVC,param)
bo.run(init_evals=30,max_iter=120)
bo6=GPGO(RF,acq3,accuracy_SVC,param)
bo6.run(init_evals=30,max_iter=120)`

Hello @camillascf, this is not fully reproduceable code. Please provide a minimal working issue example.

AML_assignment555.pdf
It includes a grid search of parameters as a comparison.


make better guide about input and coordinates of the board

Hello,
I wanted to tell you that the inputing a coordinate is confusing. So give a
better guide there
On Sun, 27 Sep 2020, 12:44 Jayanth PSY<EMAIL_ADDRESS>wrote:

@kusalhasintha https://github.com/kusalhasintha can you kindly what
this PR is about?
Did you try to open an issue?
Those changes in README.md seem unnecessary.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/j-tesla/tic-tac-toe/pull/5#issuecomment-699596547,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHMQGCZBWHYA6YZEQGBNDEDSH3Q3TANCNFSM4R3MHYOA
.

Originally posted by @kusalhasintha in https://github.com/j-tesla/tic-tac-toe/pull/5#issuecomment-699637507

On top of improving guide, I think we could also make input command more intuitive.
For example this program could ask for player modes and list the available modes for users to enter (plus exit) instead of simply print "Input command:" which isn't clear on what user could input.

@mxw7265 That'd be great!


Memoryviews work with cache_builtins=False

Now they make their own tiny cache of the relevant exceptions
at module initialization so they can still access these without
the GIL.
closes https://github.com/cython/cython/issues/3406
I guess this slightly breaks the contract of the cache_builtins option, that you should be able to monkey-patch builtin stuff (including exceptions). With that said - it's an implementation detail that it's written in Cython. If it was written in C then it'd just use the standard exception definitions in the headers and the user wouldn't be able to change them anyway.

The other approach would be to make one _err function per exception you'd like to throw.  With the GIL held in those functions, the actual exception can be looked up at runtime.
It is, however, a couple more lines than this.

I've disabled the testing for this. Changing cache_builtins affects what appears in builtin_scope which is a static object, and thus breaks subsequent tests. I can't see a way round this.

Thanks


Chef Server (and Chef Automate) don't support running behind reverse proxy using URL routing

NOTE:
We use GitHub issues to track bugs and feature requests. If you need help please post to our Mailing List or join the Chef Community Slack.

Chef Community Slack at http://community-slack.chef.io/.
Chef Mailing List https://discourse.chef.io/

(Note that I looked; https://discourse.chef.io/t/changing-the-chef-server-name-and-or-url/5180 and https://discourse.chef.io/t/changing-chef-api-url/6813 were asked by the same person 3.5 and 5 years ago and no one responded)
Chef Server Version
[What version of Chef Server? Any plugins? Are you using Chef Backend? What versions?]
Chef Server, Chef Automate (latest versions as of Feb 2019)
Platform Details
[Operating system distribution and release version. Cloud provider and instance type if running in the cloud. Total/Free RAM and disk space. Is this running in a container?]
Running on RHEL / CentOS on EC2 instances on AWS
Configuration
[Standalone, Tiered, HA, Chef Backend? New install or upgrade? Ideally include your /etc/opscode/chef-server.rb or otherwise provide details of changes from the default]
Chef and Chef Automate are both running as non-HA
Scenario:
[What you are trying to achieve and you can't?]
We have a reverse proxy that only supports URL based proxying (i.e. we'd like https://myserver/chef to proxy to https://chef.internal and https://myserver/automate to proxy to https://automate.internal).  There does not appear to be any documentation on how to do this; unlike gitlab or other tools where you can either specifcy an external URL or otherwise specify a subdirectory for the application to listen for, chef and chefautomate appear to only support a hostname (FQDN).
Steps to Reproduce:
[If you are filing an issue what are the things we need to do in order to repro your problem? How are you using this cookbook or any resources it includes?]
Using stock configuration
Expected Result:
[What are you expecting to happen as the consequence of above reproduction steps?]
https://myexternalproxy/chef brings up the chef login page and all URL's are of the form /chef/ instead of just /
Actual Result:
[What actually happens after the reproduction steps? Include the error output or a link to a gist if possible.]
Fails miserably; redirects to /

I assume that  https://myserver/automate likewise doesn't forward correctly?
If I understand things correctly you would like to access the web interface through this proxy. So chef stuff would go to chef-manage, and automate stuff would go to the automate server. I'm not sure where we stand with the new automate, but I suspect that chef-manage has some baked in assumptions about the the distinction between the host and resource portion of the URI.
So it sounds like any changes needed are going to be in the chef-manage and automate projects and so we'll need to work with them. I know there's no good way to file an issue against them, so for now we'll track this here until we sort this out in more detail.
If you want to also hit the chef server API (e.g. you want chef client/knife runs against that proxy) that will require a bit more work. The signing protocol signs the resource (e.g. chef) as part of the request, so stripping the resource will invalidate the signature, and fail.

That is correct; I was not able to get automate to support forwarding from /automate to a bare /.  It looks like automate-ui portion should be amenable (since it's running Angular and has a base-href flag, but I didn't see any knobs to change it through config.toml ... as an aside, it would be nice if updates to chef-automate created a latest sym link (so e.g. I could write a chef recipe that can count on /hab/svc/automate-ui/latest/config/whatever to be the file I want to update via chef)) but other pieces were more problematic.  Of course, one of the pieces that was most problematic was the Dex stuff (which not coincidentally I'm substantially less familiar with to know if there are any easy knobs to tweak things) and even after reverting things so automate is /, I'm still having issues with switching tabs for 60 seconds == forced to log back in, which I was initially attributing to bad interactions between applications (e.g. some other application was setting a cookie for / with the same name as chef-automate and so switching between them overwrites the other's cookie) ... so maybe Automate could be made to work.  Good to know re: chef server / knife things.
Regarding


enhancement: visual indication of console output on debug node

It's hard to tell which debug node is writing output to the console.
Would it be possible to add a little icon to the outside of the debug node, to show when it's in "debug tab and console" output mode?

just a static indication presumably ?
If you have 2 then you would still have to work it out... but...

At most it would be something prepended to the label, as we do for repeating inject nodes.

(yup that's where I was heading) - or appended

yes, absolutely fine… just so you don’t have to open each one to see if it’s the one that’s spewing content

On 22 Dec 2016, at 09:49, Dave Conway-Jones<EMAIL_ADDRESS>wrote:
(yup that's where I was heading) - or appended
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/node-red/node-red/issues/1094#issuecomment-268761258, or mute the thread https://github.com/notifications/unsubscribe-auth/ACkaKvIpT6R31_X7sEbUiuLWYaY_uJ3bks5rKkekgaJpZM4LTxhM.


Interactive RigidObject python tutorial page

Motivation and Context
Added a much needed tutorial for rigid object interactivity through the python Simulator API.
How Has This Been Tested
New tutorial code is run in tests/test_examples.py.
Types of changes

[x] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have completed my CLA (see CONTRIBUTING)
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

I am unable to build documentation:
dbatra at dbatra-mbp in ~/projects/habitat-sim/docs on rigid-object-tutorial
$ ./build-public.sh
...
...
Traceback (most recent call last):
  File "./m.css/documentation/python.py", line 2636, in <module>
    module = SourceFileLoader(name, args.conf).load_module()
  File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 823, in load_module
  File "<frozen importlib._bootstrap_external>", line 682, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "conf-public.py", line 9, in <module>
    from conf import *
  File "/Users/dbatra/projects/habitat-sim/docs/conf.py", line 17, in <module>
    import habitat_sim  # NOQA
  File "/Users/dbatra/projects/habitat-sim/docs/../habitat_sim/__init__.py", line 16, in <module>
    from habitat_sim.simulator import *
  File "/Users/dbatra/projects/habitat-sim/docs/../habitat_sim/simulator.py", line 21, in <module>
    from habitat_sim.physics import MotionType
  File "/Users/dbatra/projects/habitat-sim/docs/../habitat_sim/physics.py", line 5, in <module>
    from habitat_sim._ext.habitat_sim_bindings import MotionType, VelocityControl
ImportError: cannot import name 'VelocityControl'

Reviewing this PR was like unwrapping a basket of gifts. So many goodies.


Memory leak with tf.data.Dataset.map

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18-04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): tf-nigthtly
Python version: 3.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:

Describe the current behavior
Test is failing if line with dataset.map is not commented.
It shows the garbage collection is not working when using even a simple function with tf.data.Dataset.map
import tensorflow as tf
from tensorflow.python.framework.test_util import assert_no_garbage_created

class TestStandardTrainer(tf.test.TestCase):
  @assert_no_garbage_created
  def test_dataset_map(self):
      data_tensor = {'x': tf.constant([1, 2, 3], dtype=tf.float32),
                     'y': tf.constant([4, 5, 6], dtype=tf.float32)}
      dataset = tf.data.Dataset.from_tensor_slices(data_tensor)

      # We split the data in 2
      dataset = dataset.map(lambda x: (x, x))

import unittest
unittest.main(argv=['first-arg-is-ignored'], exit=False)

Describe the expected behavior
Test should pass
Standalone code to reproduce the issue
See above.
Other info / logs
None

/cc @jsimsa

I am able to replicate this issue, please find the gist here

I will be attempting to fix the issue in the next two weeks

Was able to replicate your issue in Tensorflow 2.5, please find the gist here. Thanks!

I am impacted by this.

Can we extend the test we have to cover this case?
https://github.com/tensorflow/tensorflow/blob/a07eb63a319efcc02c60f98c798453d2b21af785/tensorflow/python/data/kernel_tests/memory_cleanup_test.py#L161-L171

Perhaps we can also package inputs to a dictionary, run some operations on them, then unpackage as I have done in my test:
https://github.com/tensorflow/tensorflow/issues/43969#issuecomment-1170321569

@jnd77 This issue seems to be fixed using TF v2.13, please find the gist here. Thank you!

I ran the gist, but the test still seems to be failing with 2.13.0 ...

@jnd77,
I tried to execute the mentioned code on tensorflow v2.14 on both CPU and GPU & observed that the code was not failing with the Assertion error which happened on tensorflow v2.13. Kindly find the gist of here and let me provide the opinion on the same. Thank you!

@tilakrayal looks good to me. Thanks.

@jnd77,
Glad the issue got resolved. Could you please feel free to move this issue to closed status. Thank you!


Youtube age-gated video error

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2020.09.20
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dl -v -F https://www.youtube.com/watch?v=R89f5YMT-QA
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.youtube.com/watch?v=R89f5YMT-QA']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.09.20
[debug] Python version 2.7.16 (CPython) - Darwin-18.7.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube] R89f5YMT-QA: Downloading webpage
[youtube] R89f5YMT-QA: Downloading embed webpage
[youtube] R89f5YMT-QA: Refetching age-gated info webpage
ERROR: Watch this video on YouTube. Playback on other websites has been disabled by the video owner.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2231, in _real_extract
    raise ExtractorError(error_message, expected=True)
ExtractorError: Watch this video on YouTube. Playback on other websites has been disabled by the video owner.

Description

Looks like Youtube has change behavior for age-gated video. Now, youtube-dl returns ERROR: Watch this video on YouTube. Playback on other websites has been disabled by the video owner. This did not happen before.
Passing cookies, login and user-agent did not help.

Seems like I've found a solution!
20 days ago, on Sep 6, 2020 youtube-dl main repo had a commit related to age-gated content.
ytdl-org@16ee69c
So, in youtube_dl/extractor/youtube.py
I just removed the newly added 2 lines and returned the old one:
if re.search(r'player-age-gate-content">', video_webpage) is not None:
This way we can not by-pass geo restriction, but at least it works with --cookies parameter.
Worked for links like:
https://www.youtube.com/watch?v=MxCRYXg6hLs
https://www.youtube.com/watch?v=fh2cDKyFdyU

Also see https://github.com/ytdl-org/youtube-dl/issues/26652

Issue should be closed, URL is dead

Now, private video.


Mirror functionality broken

Gitea version (or commit ref): 1.0.1 (official release)
Git version: 2.7.4
Operating system: Ubuntu 16.04.01
Database (use [x]):

[ ] PostgreSQL
[X] MySQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[X] No (try.gitea is down, and I also use private repo)
[ ] Not relevant

Log gist: Not providing in case there is sensitive date. This should be easy to repro though.

Description
Hello all. It appears that sometime between the last gogs release and the latest gitea release the mirror functionality has been broken. I've noticed that none of my mirror repos have been syncing as per the 8H schedule. I can sync manually, but even after enabling and disabling sync frequencies nothing seems to have changed.
In addition, there appears to be a UI bug. When I added a second instance of my mirror repo, it gets put in the list of Repositories rather than Mirrors.
Repro
I created a test account on try.gitea.io, username test33 and password test33. I set up a mirror of my dotfiles repo. Right now https://try.gitea.io is returning a 404, so I am unable co confirm. However, is someone can set up a mirror on their instance and see if they encounter the two issues (not syncing, and wrong UI) then we can get to the bottom of this.
I will try to find time later this week to set up a second instance and see if mirrors are broken on a clean install. Perhaps the gogs -> gitea migration broke some mirror hook. To be safe, I cleared all the update hooks and updated SSH keys.

I thought this was the case to
however you just have to enable the cron in the app.ini and the updating works.
; Update mirrors
[cron.update_mirrors]
SCHEDULE = @every 10m

; Repository health check
[cron.repo_health_check]
SCHEDULE = @every 24h
TIMEOUT = 60s
; Arguments for command 'git fsck', e.g. "--unreachable --tags"
; see more on http://git-scm.com/docs/git-fsck/1.7.5
ARGS =

[cron]
ENABLED = true
RUN_AT_START = true

Thank you! I had no idea. Is this part of the upgrade flow? I can imagine many gogs users who are used to mirrors working who don't know about this when they migrate.

I am not positive.  Both of the app.ini files seem to have a "default" of enabled, so I am not positive where the disconnect happened.

Sounds good. Closing for now, people can reopen if they stumble upon this issue as well. Thank you for helping with the fix.

I think the problem still exist. Done a clean install (1.0.0+339-g5bd22a2 + SQLite) and added a mirror. No cron jobs showing under monitoring and repo is not updated.
After adding values to custom app.ini it works. Looks like default values are not working.

Wrong default value seems is an issue.


⚠ WSINGRESOS - OXXO has degraded performance

In c5f1668, WSINGRESOS - OXXO (https://srvshyweb.yucatan.gob.mx/cgi-bin/wspd_cgi.sh/WService=wsingresos/internet/compartido/wspagosoxxo.r) experienced degraded performance:

HTTP code: 200
Response time: 2371 ms

Resolved: WSINGRESOS - OXXO performance has improved in d3a217e after 4 hours, 12 minutes.


Name constructor functions

All our prototype functions are called constructor as you can see it in 1. Renaming them to the real class/object name would make debugging easier (in Chrome, you see only constructor, but you have no idea what object that really is).
@kkujala, is that something you want to tackle?

Sure, that sounds easy. Any preferences on naming?

See #793. I hope this is what you are looking for. I can continue with the other constructors later on.

I guess I should re-post this here:
This will be nice. One thought though, do we just want to name the function the same as the name of the "class"? This would create something that looks more like what coffee script generates when doing classes.
Also, should we decide on a standard way to do classes and put it on the wiki?

I guess the name of the class is on three levels:
e.g.

var RefSet = (function refSet() {
  function refSetConstructor() {
    this.dict = {};
  }
  ...
  return refSetConstructor;
}

vs.

var RefSet = (function RefSet() {
  function RefSet() {
    this.dict = {};
  }
  ...
  return RefSet;
}

Align all of them or some of them?

@kkujala, that patch already looks awesome. Thanks for taking it up :) And sorry, that we have some discussion going after you've created the patch about the details :/

var RefSet = (function RefSet() {
function RefSet() {
this.dict = {};
}
...
return RefSet;
}

I think we should name the "class" the same way it's exposed. That means naming it RefSet and not RefSetConstructor, because that might confuse people as we speak about the same thing. If you're debugging, you will see the string RefSet then, which is also smaller.
However, we might want to change the closure-function name to:
var RefSet = (function RefSetConstructor() {

That basically that we don't have to functions called the same way.

Also, should we decide on a standard way to do classes and put it on the wiki?

+1.

np. I updated the patch accordingly.

My earlier pull request was closed as the magnitude of the task/style became too unclear. I think we should take step back and only address your initial issue.
I propose we only name the constructors and not touch anything else.

var RefSet = (function refSet() {
  function RefSetConstructor() {
    this.dict = {};
  }
  ...
  return RefSetConstructor;
})();

This would still keep the existing style and help in debugging.
If this is ok then I can make the changes and update the wiki. I suppose if this is not wanted at all then this issue could be closed and wiki page deleted.

I'd like it too look like this:

var RefSet = (function refSetClosure() {
  function RefSet() {
    this.dict = {};
  }
  ...
  return RefSet;
})();

because this will get the object name right on WebKit I guess.

That is exactly what is in the latest patch of #793. It can be merged anytime if you want to.

Now that this has landed and we have https://github.com/mozilla/pdf.js/wiki/Style-Guide I'm going to close this.


correct default log4j2 config file path in helm chart

label: bugfix

Codecov Report

Merging #11707 (8aec49c) into master (cf8fd93) will decrease coverage by 48.62%.
Report is 2 commits behind head on master.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master   #11707       +/-   ##
=============================================
- Coverage     63.04%   14.43%   -48.62%
+ Complexity     1117      201      -916
=============================================
  Files          2342     2342
  Lines        125780   125778        -2
  Branches      19334    19334
=============================================
- Hits          79301    18150    -61151
- Misses        40828   106091    +65263
+ Partials       5651     1537     -4114

Flag
Coverage Δ

integration
?

integration1
?

integration2
?

java-11
?

java-17
?

java-20
14.43% <ø> (-48.47%)
:arrow_down:

temurin
14.43% <ø> (-48.62%)
:arrow_down:

unittests
14.43% <ø> (-48.62%)
:arrow_down:

unittests1
?

unittests2
14.43% <ø> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1513 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more