NetKVM: Cap max length of the packet with NB size

It was noticed that as of build 20241 the sum of lengths of MDLs in NET_BUFFER
can be greater than the size of NET_BUFFER itself. As the result, the size of the
packet can be calculated incorrectly.
Instead of using the sum of MDLs lengths, we can cap the max length with the size
of NET_BUFFER.
Bug: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/583
This PR is to also run HLK tests by CI.

@sb-ntnx If I'm not mistaken we never use MDL length as number of bytes to copy, it is used as limit to never copy more than that when the number of bytes to copy never comes from the MDL length.  As far as I remember from the history of the driver, the situation when the NET_BUFFER is smaller than the MDL length is very common. Can you please provide some description of the exact call chain and parameters that ensures that this change does some read job and not always-true check (probably with some logs?). I see the link to the bug but do not see any comment that this fix  solves this bug.

@ybendito: there is an email thread with the subject "Connection issues with NetKVM + Windows Server 2022 combination #583" where @YanVugenfirer added you to. It has all details. Let me copy it here.
If you look at this picture that I posted in the bug:
https://user-images.githubusercontent.com/56913349/122566816-43e16600-d048-11eb-8b84-d961fd914eb4.png
You can see that the DNS packet on the receiver has extra padding at the end (trailer) and it seems like these two bytes are added by the sender when a packet is put into vring. For example, this is a snippet from live debug session (see attached picture):
0: kd> !db 0x7d4d2008 L4e
#7d4d2008 01 00 00 00 00 00 22 00-06 00 aa cc 50 6b 8d 9b ......".....Pk..
#7d4d2018 59 e6 50 6b 8d c0 ca 18-08 00 45 00 00 32 18 fe Y.Pk......E..2..
#7d4d2028 00 00 80 11 97 11 0a 42-3b 18 0a 42 3b 10 e7 8c .......B;..B;...
#7d4d2038 00 35 00 1e 8a db 17 3f-01 00 00 01 00 00 00 00 .5.....?........
#7d4d2048 00 00 01 74 02 63 6f 00-00 01 00 01 8d 5f ...t.co......_..
0: kd> k
# Child-SP          RetAddr           Call Site
00 ffff858f`9c0c1c80 fffff804`307d1b1d netkvm!CTXDescriptor::Enqueue+0x41 [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis_virtqueue.cpp @ 354]
01 ffff858f`9c0c1cd0 fffff804`307cfe14 netkvm!CTXVirtQueue::SubmitPacket+0xd5 [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis_virtqueue.cpp @ 246]
02 ffff858f`9c0c1d00 fffff804`307ceb7a netkvm!CParaNdisTX::SendMapped+0xd4 [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis_tx.cpp @ 593]
03 (Inline Function) --------`-------- netkvm!CParaNdisTX::DoPendingTasks::__l2::<lambda_5f99f3cd9e1c0a03d928425271afce82>::operator()+0x36 [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis_tx.cpp @ 704]
04 (Inline Function) --------`-------- netkvm!CParaNdisTX::DoWithTXLock+0x40 [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis-tx.h @ 221]
05 ffff858f`9c0c1d60 fffff804`307ceec3 netkvm!CParaNdisTX::DoPendingTasks+0x92 [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis_tx.cpp @ 693]
06 ffff858f`9c0c1dd0 fffff804`307cfa77 netkvm!CParaNdisTX::NBLMappingDone+0x3f [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis_tx.cpp @ 403]
07 ffff858f`9c0c1e00 fffff804`324ba0ac netkvm!CNBL::RegisterMappedNB+0x27 [e:\build\kvm-guest-drivers-windows\netkvm\common\parandis_tx.cpp @ 90]
08 ffff858f`9c0c1e30 fffff804`325aa696 nt!HalpConstructScatterGatherListDmaThin+0x14c
09 ffff858f`9c0c1e90 fffff804`33d15384 nt!HalBuildScatterGatherListDmaThin+0xf6
0a ffff858f`9c0c1ee0 ffffd38d`92dae800 0xfffff804`33d15384
0b ffff858f`9c0c1ee8 ffffd38d`98b461a0 0xffffd38d`92dae800
0c ffff858f`9c0c1ef0 00000000`00000002 0xffffd38d`98b461a0
0d ffff858f`9c0c1ef8 00000000`00000066 0x2
0e ffff858f`9c0c1f00 00000000`00000066 0x66
0f ffff858f`9c0c1f08 fffff804`307d74b0 0x66
10 ffff858f`9c0c1f10 ffffd38d`98efa428 netkvm!ProcessSGListHandler [e:\build\kvm-guest-drivers-windows\netkvm\wlh\parandis6_impl.cpp @ 1094]
11 ffff858f`9c0c1f18 00000000`00000001 0xffffd38d`98efa428
12 ffff858f`9c0c1f20 ffffd38d`9457e050 0x1
13 ffff858f`9c0c1f28 00000000`000003e8 0xffffd38d`9457e050
14 ffff858f`9c0c1f30 00000000`00000000 0x3e8

Last two bytes it is the trailer I saw on the tap interfaces of VMs (sender and receiver). These trailer bytes can be seen on the sender and the receiver tap interfaces on AHV and vanilla KVM.
So this length is coming from CNB::Copy, where I added some additional logging is it is quite hard to catch in in the live debugging session:
ULONG CNB::Copy(PVOID Dst, ULONG Length) const
{
    ULONG CurrOffset = NET_BUFFER_CURRENT_MDL_OFFSET(m_NB);
    ULONG Copied = 0;

    for (PMDL CurrMDL = NET_BUFFER_CURRENT_MDL(m_NB);
         CurrMDL != nullptr && Copied < Length;
         CurrMDL = CurrMDL->Next)
    {
        ULONG CurrLen;
        PVOID CurrAddr;

#if NDIS_SUPPORT_NDIS620
        NdisQueryMdl(CurrMDL, &CurrAddr, &CurrLen, MM_PAGE_PRIORITY(LowPagePriority | MdlMappingNoExecute));
#else
        NdisQueryMdl(CurrMDL, &CurrAddr, &CurrLen, MM_PAGE_PRIORITY(LowPagePriority));
#endif

        if (CurrAddr == nullptr)
        {
            break;
        }
        DPrintf(1, "[%s] Length: 0x%x, Copied: 0x%x, CurrLen: 0x%x, CurrLen (fixed): 0x%x\n", __FUNCTION__, Length, Copied, CurrLen, min(CurrLen - CurrOffset, Length - Copied));
        CurrLen = min(CurrLen - CurrOffset, Length - Copied);

        NdisMoveMemory(RtlOffsetToPointer(Dst, Copied),
                       RtlOffsetToPointer(CurrAddr, CurrOffset),
                       CurrLen);

        Copied += CurrLen;
        CurrOffset = 0;
    }

    return Copied;
}

With this additional logging, I can see that the length is calculated from two MDLs:
00000033     netkvm.sys   1288   3852   1      33     07\15\2021-07:12:12:157       CNB::Copy[CNB::Copy] Length: 0xfec, Copied: 0x0, CurrLen: 0x50, CurrLen (fixed): 0x2a
00000034     netkvm.sys   1288   3852   1      34     07\15\2021-07:12:12:157       CNB::Copy[CNB::Copy] Length: 0xfec, Copied: 0x2a, CurrLen: 0x18, CurrLen (fixed): 0x18
00000035     netkvm.sys   1288   3852   1      35     07\15\2021-07:12:12:157       QualifyIpPacketip_version 4, ipHeaderSize 20, protocol 17, ip_src 183b420a, ip_dst 428420a, iplen 50, L2 payload length 64
00000036     netkvm.sys   1288   3852   1      36     07\15\2021-07:12:12:157       PrintOutParsingResult[CNB::QueryL4HeaderOffset] UDPv4 packet IPCS not tested<NULL>, checksum -<NULL>
00000037     netkvm.sys   1288   3852   1      37     07\15\2021-07:12:12:157       QualifyIpPacketip_version 4, ipHeaderSize 20, protocol 17, ip_src 183b420a, ip_dst 428420a, iplen 50, L2 payload length 50
00000038     netkvm.sys   1288   3852   1      38     07\15\2021-07:12:12:157       PrintOutParsingResult[CNB::DoIPHdrCSO] UDPv4 packet IPCS Bad(fixed), checksum -<NULL>
00000039     netkvm.sys   1288   3852   1      39     07\15\2021-07:12:12:157       CNB::PrepareOffloads[CNB::PrepareOffloads] EthPayloadLength 0x32, L4HeaderOffset 0x22
00000040     netkvm.sys   1288   3852   1      40     07\15\2021-07:12:12:157       CNB::BindToDescriptor[CNB::BindToDescriptor] HeadersLength 0x42                       <<<< sum of CurrLen of MDLs

As you can see that sum of MDLs size gives us 0x42, while the expected packet length is 0x40.
As I’ve mentioned earlier, the issue does not reproduce if we use nslookup.exe and the difference is in the length of the second MDL:
00000104     netkvm.sys   2640   5000   0      30     07\15\2021-06:50:03:688       CNB::Copy[CNB::Copy] CurrLen: 0x50, CurrLen (fixed): 0x2a
00000105     netkvm.sys   2640   5000   0      31     07\15\2021-06:50:03:688       CNB::Copy[CNB::Copy] CurrLen: 0x16, CurrLen (fixed): 0x16
00000106     netkvm.sys   2640   5000   0      32     07\15\2021-06:50:03:688       QualifyIpPacketip_version 4, ipHeaderSize 20, protocol 17, ip_src 183b420a, ip_dst 428420a, iplen 50, L2 payload length 64
00000107     netkvm.sys   2640   5000   0      33     07\15\2021-06:50:03:688       PrintOutParsingResult[CNB::QueryL4HeaderOffset] UDPv4 packet IPCS not tested<NULL>, checksum -<NULL>
00000108     netkvm.sys   2640   5000   0      34     07\15\2021-06:50:03:688       QualifyIpPacketip_version 4, ipHeaderSize 20, protocol 17, ip_src 183b420a, ip_dst 428420a, iplen 50, L2 payload length 50
00000109     netkvm.sys   2640   5000   0      35     07\15\2021-06:50:03:688       PrintOutParsingResult[CNB::DoIPHdrCSO] UDPv4 packet IPCS Bad(fixed), checksum -<NULL>
00000110     netkvm.sys   2640   5000   0      36     07\15\2021-06:50:03:688       CNB::PrepareOffloads[CNB::PrepareOffloads] EthPayloadLength 32, L4HeaderOffset 22
00000111     netkvm.sys   2640   5000   0      37     07\15\2021-06:50:03:688       CNB::BindToDescriptor[CNB::BindToDescriptor] HeadersLength 40

The sum is 0x40, which is expected.
However, if we look at NET_BUFFER instead of MDLs, which this NET_BUFFER consists of, we can see that it has correct size:
00000065     netkvm.sys   1288   2800   0      65     07\16\2021-06:37:42:413       CNB::CopyNET_BUFFER_DATA_LENGTH(m_NB): 0x40, NET_BUFFER_DATA_OFFSET(m_NB): 0x26, NET_BUFFER_CURRENT_MDL_OFFSET(m_NB): 0x26
00000066     netkvm.sys   1288   2800   0      66     07\16\2021-06:37:42:413       CNB::Copy[CNB::Copy] Length: 0xfec, Copied: 0x0, CurrLen: 0x50, CurrLen (fixed): 0x2a                                       <<<< size of the first MDL in NB
00000067     netkvm.sys   1288   2800   0      67     07\16\2021-06:37:42:413       CNB::Copy[CNB::Copy] Length: 0xfec, Copied: 0x2a, CurrLen: 0x18, CurrLen (fixed): 0x18                                       <<<< size of the second MDL
00000068     netkvm.sys   1288   2800   0      68     07\16\2021-06:37:42:413       QualifyIpPacketip_version 4, ipHeaderSize 20, protocol 17, ip_src 183b420a, ip_dst 428420a, iplen 50, L2 payload length 64
00000069     netkvm.sys   1288   2800   0      69     07\16\2021-06:37:42:413       PrintOutParsingResult[CNB::QueryL4HeaderOffset] UDPv4 packet IPCS not tested<NULL>, checksum -<NULL>
00000070     netkvm.sys   1288   2800   0      70     07\16\2021-06:37:42:413       QualifyIpPacketip_version 4, ipHeaderSize 20, protocol 17, ip_src 183b420a, ip_dst 428420a, iplen 50, L2 payload length 50
00000071     netkvm.sys   1288   2800   0      71     07\16\2021-06:37:42:413       PrintOutParsingResult[CNB::DoIPHdrCSO] UDPv4 packet IPCS Bad(fixed), checksum -<NULL>

The workaround seems to be fairly simple, and I tested on several modern Windows builds, and it is working as expected:
diff --git a/NetKVM/Common/ParaNdis_TX.cpp b/NetKVM/Common/ParaNdis_TX.cpp
index 76745c37..fef062cc 100644
--- a/NetKVM/Common/ParaNdis_TX.cpp
+++ b/NetKVM/Common/ParaNdis_TX.cpp
@@ -960,6 +960,8 @@ ULONG CNB::Copy(PVOID Dst, ULONG Length) const
     ULONG CurrOffset = NET_BUFFER_CURRENT_MDL_OFFSET(m_NB);
     ULONG Copied = 0;

+    Length = min(Length, NET_BUFFER_DATA_LENGTH(m_NB));
+
     for (PMDL CurrMDL = NET_BUFFER_CURRENT_MDL(m_NB);
          CurrMDL != nullptr && Copied < Length;
          CurrMDL = CurrMDL->Next)

Memory dump with netkvm.pdb can be found here: https://nutanixinc-my.sharepoint.com/:f:/g/personal/sergey_bykov_nutanix_com/EqXDTni9aERAhPWQOqQhmZ8BmXTIcRbG2QcJcUa4KBhXpA?e=K3vG5i

@sb-ntnx I can't get the dump. "We're sorry, but  can't be found in the nutanixinc-my.sharepoint.com directory".

@ybendito: can you try this link then? https://1drv.ms/u/s!Alp51rbA-o0kmv5lMp-KqXI8dKg3tg?e=S84k9K

@ybendito: can you try this link then? https://1drv.ms/u/s!Alp51rbA-o0kmv5lMp-KqXI8dKg3tg?e=S84k9K

Yes, this one works. You're right. Thank you for digging into it.

@sb-ntnx Thanks a lot!
Waiting for CI to finish to merge.

Update from Microsoft on the bug I raised with them:
Please use the NET_BUFFER.DataOffset and NET_BUFFER.DataLength fields (or better yet, the NET_BUFFER_DATA_OFFSET and NET_BUFFER_DATA_LENGTH field accessors) for determining the length of the packet. See NET_BUFFER Structure - Windows drivers | Microsoft Docs (https://docs.microsoft.com/en-us/windows-hardware/drivers/network/net-buffer-structure)
Trailing bytes in the last MDL in the MDL chain is valid.
See related post
NET_BUFFER_DATA_LENGTH Problem | TechTalkz.com - Technology and Computer Help Forums https://www.techtalkz.com/threads/net_buffer_data_length-problem.274548/


Viddy as an importable package.

This PR should allow importing Viddy tool as a lib in other Golang projects.
Also, I've added the PreconfiguredNewViddy func to avoid additional dependencies in 3-party projects.
I suppose this is a good structure to use both:

as a library
as a binary

Ready for comments and discussions.

thanks @anvial
The project has a dependency on the TUI lib. I am planning to use viddy in a WASM project where it is cross compiled using tinygo and then run it in the browser.   So I was wondering if the project can be structured so that the TUI and the core viddy are separated ?
cc  @sachaos

@gedw99 It is interesting that you are going to run it in WASM.
I have a few questions. (I don't have any WASM experience, so sorry if I'm wrong.

What would be the use case in WASM? viddy executes a command and outputs it, but what would it output instead?
If we don't depend on the TUI library, what will we depend on instead?

Hey @anvial excellent questions and thanks for asking. Let me know if this explains things better ...

What would be the use case in WASM? viddy executes a command and outputs it, but what would it output instead?

Service workers inside a browser are essentially http servers. You can write them in golang and compile them to wasm.  So they are processes, just like you have processes on your labtop.
I am building a quasi IDE in golang, and want to use viddy in exactly the same way. This quasi IDE is in golang, and compiled to run as a GUI.
The same is also true on the server side. Checkout https://github.com/tetratelabs/wazero. You can compile your golang to wasm, and get instant starts and sandboxing. Viddy can then used with wazero to inspect what is going on with the golang / WASM running on a local or remote server.

If we don't depend on the TUI library, what will we depend on instead?

What i propose is to have viddy as a golang package. Then any GUI / TUI can import it and use it.
I am using https://github.com/gioui  to build the GUI that does what the current TUI does. It can run on Web, Desktop and Mobiles.  Its 100% golang code just like the TUI.

@gedw99 Thank you.
It is nice to remove the dependency and user can change the view.
But currently, viddy depends on rivo/tview (TUI lib) heavily.
And I think it would be hard.
I don't think removing rivo/tview should be done in this PR.
If you want to remove rivo/tview from viddy lib.
Feel free to submit the PR.

I don't think removing rivo/tview should be done in this PR.

@sachaos your right. Its best to do in later PR.
I will make an issue now for it so we can swing back to it later.

@sachaos,
what is your feedback about current PR? What do you think about changing the structure?
I also need your advice and suggestion on how to implement the --version flag, because looks like this PR has broken it...

@anvial About structure, please refer to my idea of importable package of viddy.
(This is WIP and not working, but the interface would be that.
https://github.com/sachaos/viddy/blob/feature/refactoring-viddy/pkg/viddy/viddy.go
I'm happy if you like this and follow that in this PR.
About, version and help.
Please parse os.Args by newConfig() in main.go, and handle that in main.go.
Then pass the Config struct to NewViddy().
Then the viddy lib package will no longer depends on viper.

@sachaos
https://github.com/sachaos/viddy/blob/feature/refactoring-viddy/pkg/viddy/viddy.go
Your approach looks clean and smooth. I think this is the right way to implement it.
How can we help the refactoring-viddy branch?

@anvial If you already have viddy on the juju side, making viddy importable may not be an urgent issue. In that case, I would appreciate it if you could wait for my work.
If there are other requests and it should be importable, I think it is possible to imitate only the viddy interface of that branch and leave the content unrefactored on this PR.
Then, even if I refactor later, the interface should not change, so I think that the users of the library will not be bothered.


Can't create workflow dispatch event by filename

Noticed that, GitHub API docs: Create a workflow dispatch event, let's you create dispatch event by both workflowID and Filename.
However, PR #1623  adds support for just workflowID, with CreateWorkflowDispatchEvent.
Can we split it into two funcs for each type, as that's the general implementation in other APIs?

Excellent catch, @n1lesh  - thank you!  Yes, that would be consistent and generally useful.
Let's do that, please.

This will be a breaking API change, but I think we should do it, so I've labeled it as such.


BUG - If the field is not introduced in mock data then the error is coming

created for traceability
https://github.wdf.sap.corp/fiori-elements/v4-consulting/issues/3659
If the field is not introduced in mock data then the error is coming.
Latest version of node module '<EMAIL_ADDRESS>installed.
ui5-middleware-fe-mockserver 2.1.62 - all was ok from this perspective

Fixed with 2.1.87


add detect dedecms!

add detect dedecms

Thank you!

@happytaoer Thankyou For Suggesting :) I'll surely add DEDE CMS detection in the upcoming version

Tuhinshubhra

Note that get_headers WILL follow redirections (HTTP redirections). New headers will be appended to the array if $format=0. If $format=1 each redundant header will be an array of multiple values, one for each redirection.
http://php.net/manual/zh/function.get-headers.php


PHOENIX-5773 Index tool output tables should support multiple simulta…

…neous rebuilds

A small test to cover the changed part of the code would be highly appreciated. Thanks.

@swaroopak, @gjacoby126 has offered to extract the code for index tool tables out and write unit tests for it. As part of that effort, the unit tests for index tool table row keys can be added.

@kadirozde - I offered (and am still offering) to split out the code into a separate class and write read / field parsing logic in a separate JIRA, but I believe @swaroopak 's request for a test to cover the write logic you've changed is still needed for this JIRA.

@kadirozde - I offered (and am still offering) to split out the code into a separate class and write read / field parsing logic in a separate JIRA, but I believe @swaroopak 's request for a test to cover the write logic you've changed is still needed for this JIRA.

@gjacoby126, the change is covered by many existing tests already. You cannot get verification counters correct if the key is not constructed properly. As I said before, this JIRA does not offer anything for parsing or extracting subfields within a row key.

@kadirozde - I offered (and am still offering) to split out the code into a separate class and write read / field parsing logic in a separate JIRA, but I believe @swaroopak 's request for a test to cover the write logic you've changed is still needed for this JIRA.

@gjacoby126, the change is covered by many existing tests already. You cannot get verification counters correct if the key is not constructed properly. As I said before, this JIRA does not offer anything for parsing or extracting subfields within a row key.

@kadirozde - I offered (and am still offering) to split out the code into a separate class and write read / field parsing logic in a separate JIRA, but I believe @swaroopak 's request for a test to cover the write logic you've changed is still needed for this JIRA.

@gjacoby126, the change is covered by many existing tests already. You cannot get verification counters correct if the key is not constructed properly. As I said before, this JIRA does not offer anything for parsing or extracting subfields within a row key.

Okay. I have added testing for row keys.


CreateChatUsingBlob doesn't work with new blob skype gives. Old group chats with a long blob still work

Apparently skype has another way to present blobs, when i looked in an older chat the blob was significantly longer than when creating a new groups chat. It works with the longer blob but the new shorter blob doesnt work. Did i do something wrong?

Got the same problem.
Want to mention that with old group chat it's only working with invites. People cant join without invites so no browser link working or skype web UI...
Same problem here:
https://github.com/boamaod/skype2irc/issues/7

Is that the problem with the chats need to be p2p chats? There is an added section about that in the readme.


Improve code sharing URL generator

I have noticed that the share URL generator just inserts the code and the compiler settings in the URL.
That works, but for large code snippets and sites with limitted URL lengths (Yes Twitter/stackoverflow, I'm talking about you...) that URLs cannot be shared easily/directly.
It would be awesome if the web generates better (shorter) urls. I think that URLs are generated in that way because GCC explorer server does not host the sourcecode, only provides a compiler frontend. But would be great if the server push the code automatically to a web like pastebin.com, and when the URL is accessed, that code is pulled from that web and written in gcc-explorer automatically.

I assume this was fixed by #47?

Closing; I believe it's fixed...


Mixture: Add mixture update job per user's request (market Mixture with genereate UI)

What about situation when mixture goes outdated? That means aroma/booster/base change.
Also there is need to keep track of refreshed mixtures to show user a progress (there could eventually exists more update jobs per one mixture).

Done.


ELM-1312 Add AWS transfer workflow

@pricemg Terraform plan evalaution detected changes to resources that require approval from a member of @ministryofjustice/modernisation-platform

@pricemg Terraform plan evalaution detected changes to resources that require approval from a member of @ministryofjustice/modernisation-platform

@pricemg Terraform plan evalaution detected changes to resources that require approval from a member of @ministryofjustice/modernisation-platform


use RoutesBuilder interface in CamelTestSupport for more flexibility

There's nothing in CamelTestSupport that requires specific functionality of the Java DSL RouteBuilder.  Making it use the generic interface gives users more freedom to build test routes in other ways.

Can I get some feedback on this?  It's a pretty minor change.

Thanks for the PR this is now being merged.


Remove dead link to unixstickers.com

The link does not link Requests stickers as the text suggests.

Codecov Report

Merging #4804 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4804   +/-   ##
=======================================
  Coverage   66.77%   66.77%
=======================================
  Files          15       15
  Lines        1568     1568
=======================================
  Hits         1047     1047
  Misses        521      521

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 207e83f...e71396a. Read the comment docs.


Sailsjs v1

Hi
Do you have a plan to migrate your solution to sailsjs version 1?
Thanks

Yes I will, when it hits npm. npm is still on 0.12.


Cannot use TokenType.Aad since 0 is equal to false

TokenType.Aad represents a 0 and this prop cannot be passed as a string.
0 when evaluated by JS is false.
Inside component implementation there is a function called getConfigFromProps that validates tokenType sent to the component, as it follows:
tokenType: tokenType || TokenType.Embed,
This always fails if left side of the expression is 0.
Maybe you could change Aad to 2 or allow to pass string as tokenType param.
Thanks.

same problem here.
this also happens with Permissions.Read (= 0).
I patched using ?? (nullish coalescing) instead of the || (logical OR).
permissions: permissions ?? Permissions.All,
tokenType: tokenType ?? TokenType.Embed,
type: embedType ?? 'report'
but this is a relatively new feature in ES, so it depends on which versions of node/typescript/target es/etc you are using.
if you can't use ??, you may use something like:
permissions: (permissions != (null || undefined)) ? permissions : Permissions.All,
tokenType: (tokenType != (null || undefined)) ? tokenType : TokenType.Embed,
type: (embedType != (null || undefined)) ? embedType : 'report'


guiPressedHook();

I think it would be nice to have the ability to detect if/when the player touches the D-pad (or other default GUI buttons).
I know this is not the best example but somthing like this could be pretty usefull.
function guiPressedHook(buttonType, actionType){
      If(buttonType == "LEFT" && actionType == "DOWN" ){
           clientMessage("You hit the left button! Now you will die.");
           //code to kill the player
      }
      If(buttonType == "RIGHT" && actionType == "UP" ){
           clientMessage("You stopped pressing the right button! Now you will die.");
           //code to kill the player
      }
 }

@TDS200 @zhuowei nice suggestions also a function when you hold the screen (long press)

@officialmcpro32 better,
ModPE.getOnTouch(x,y);
how to use?:
if(ModPE.getOnTouch(10,10)==ScreenType.IN_GAME){
print("you pressing in left/right, idk");
}}

guiPressedHook dont make sense :P

@ISAAC300 There's one problem with that, how will you know exactly where the D-pad is on different screen sizes/densities?

ok,
function modTick(){
ModPE.showTipMessage(ModPE.getOnTouch(x,y));
}
@TDS200
easy XD


Wondershare Filmora Crack 14.0.7 + Key Full Download [Latest]

Wondershare Filmora Crack 14.0.7 + Key Full Download [Latest]
Wondershare Filmora Crack is an very impressive video editing software. It allows you to edit and convert videos and share them on a variety of different hosting sites. Filmora 14 Crack Free Download comes with a rich library of video effects, graphic elements, transitions and titles, which can be added to your project to create some great videos.
Download Now

.
Direct Download Link
.
.
.
.
.
.

DIRECT DOWNLOAD SETUP
.
.
.
.


Styling with tailwind css configuration

Hello,
It is necessary to add information about installing the package https://www.npmjs.com/package/mini-svg-data-uri. In addition, tailwind's styles importing address<EMAIL_ADDRESS>does not work. It must be replaced with an address<EMAIL_ADDRESS> :)

Thanks


assistant response shouldn't be allowed to play concurrently

Currently if pressing the button while the assistant response is playing, a new request will be recorded and there is a possibility that the response audio will play on top of each other.
We should either interrupt the assistant response playback or queue them.
@Fleker what do you think?

I think if we interrupt the Assistant mid-playback, it should remove audio from the previous response. That's the behavior Google Home and other devices use.
We could put each audio sample into a queue perhaps and then wipe it. Right now we call mAudioTrack.write. Is there a way to stop the audio track or clear its internal queue?

Sound possible:
https://developer.android.com/reference/android/media/AudioTrack.html#stop()
For an immediate stop, use pause(), followed by flush() to discard audio data that hasn't been played back yet.


Inline gif query - Bad Request: RESULTS_TOO_MUCH (400)

Tried to load 200 mp4gifs to tb.Mpeg4GifResult and got this error, how I can avoid it and load all 200 gifs?

It says no more than 50
https://core.telegram.org/bots/api#answerinlinequery


Added parallels builder to Windows 2012 R2

I don't have a licensed copy of Parallels. Since this is about 2 months stale, can you run a quick build against the latest and just confirm it works (with windows updates and all that)

For what it's worth, I've just successfully built a Windows 2012 R2 image using the parallels details here, and it's now happily running a Vagrant box. Looks like it still works :)

I don't have the license anymore, so I can't test it out @kensykora but it looks like @tbbuck got it properly working.

It's been a few hours of running now and it's still working as expected. I think you'd need a reminder to install the vagrant parallels plugin (https://github.com/Parallels/vagrant-parallels) - otherwise all looks very good :)
(and thanks for your work on this, you've saved me some pain!)


Add check before defining custom v-slot element

This prevents errors when multiple vampire-slot components from different libraries land on the same page at the same time.
@daniel-nagy I can't get any of the tests to pass on a fresh clone, but this is a pretty common pattern I've seen when defining custom elements. Thoughts on including this?

I can’t get any of the tests to pass on a fresh clone.

This is the reason https://github.com/webcomponents/custom-elements/pull/180
You’ll need to open the custom elements polyfill and edit the minified code to change a instanceof Object to typeof a === 'object' and then the tests should work.


Removal

Is it possible to remove the Hp and HP+ from the cheat as its useless and just have overheal

Sure I can get rid of HP and combine them to one HP+

Sure I can get rid of HP and combine them to one HP+

alright thanks


PlantUML include resolver is not available in the Ruby extension

Hi,
I am trying to create a set of libraries of files that are included in most of my plantuml diagrams and I'm having a hard time figuring out why kroki-plantuml-include-paths is not working
My setup:
Ubuntu 20.04 on which I have

a local kroki server started with the follwoing docker compose file:

version: "3"
services:
  kroki:
    image: yuzutech/kroki
    environment:
      - KROKI_SAFE_MODE=UNSAFE
      - KROKI_PLANTUML_ALLOW_INCLUDE=true
    ports:
      - "8000:8000"

the official docker container which I started with

docker run -it --rm -v $(pwd):/documents asciidoctor/docker-asciidoctor

and to which I installed the gem using
gem install asciidoctor-kroki

and a small project

lib.puml
@startuml lib

skinparam sequence {
ActorBorderColor Black
LifeLineBorderColor black
ParticipantBorderColor black
ArrowColor Black
LifeLineBackgroundColor black
}

@enduml

seq.puml
@startuml

!include ./lib.puml

Actor A
Actor B

A -> B: test
A <-- B: response
@enduml

test.adoc
:doctype: book
:backend-html5:
:doctitle: test

:kroki-server-url: http://<IP_ADDRESS>:8000
:kroki-fetch-diagram:
:kroki-plantuml-include-paths: {docdir}
:imagesdir: images

{kroki-plantuml-include-paths}

[[section-adoc]]

= adoc

[plantuml]
....
include::seq.puml[]
....

all the files are in the same directory(mounted to /documents to docker)
bash-5.1# asciidoctor -r asciidoctor-kroki test.adoc
asciidoctor: FAILED: /documents/test.adoc: Failed to load AsciiDoc document - 400 Bad Request
  Use --trace for backtrace

Kroki log shows
{"timestamp":"1620972621879","level":"ERROR","thread":"vert.x-eventloop-thread-1","mdc":{"error_message":"cannot include ./lib.puml (line: 2)","path":"/plantuml/svg/eNpzKC5JLCopzc3h4lLMzEvOKU1JVdDTz8lM0isACzoml-QXKThCaSeggIKunYKTlUJJanEJkGOjqwviFaUWF-TnFadyOaTmpQA1AgCk_hqg","method":"GET","action":"error","error_code":"400","failure_class_name":"io.kroki.server.error.BadRequestException","user_agent":"Ruby"},"logger":"io.kroki.server.error.ErrorHandler","message":"An error occurred","context":"default","exception":"io.kroki.server.error.BadRequestException: cannot include ./lib.puml (line: 2)\n\tat io.kroki.server.service.Plantuml.convert(Plantuml.java:269)\n\tat io.kroki.server.service.Plantuml.lambda$convert$5(Plantuml.java:235)\n\tat io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:179)\n\tat io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:96)\n\tat io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:177)\n\tat io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n"}

using :kroki-plantuml-include: {docdir}/lib.puml worked but this is not enough for me as I have different includes for different files.
removing the include also generates the document successfully

My project is more complicated than this, with a more elaborated folder structure but I strip it down to the simplest form and not even this works.
What am I doing wrong?

kroki-plantuml-include-paths is currently only supported when using the JavaScript extension.
So this feature needs to be implemented in the Ruby extension as well.

I'm experiencing similar issue. I'm generating the  docs with Antora ,myDiagramm.puml looks like below :
@startuml
!include globals.iuml
....

@enduml

and globals.iuml resides in the same folder as myDiagramm.puml,  which is partials/uml
When executing Antora, I'm getting
Skipping preprocessing of PlantUML include, because reading the referenced local file 'globals.iuml' caused an error:
Error: ENOENT: no such file or directory, open 'globals.iuml'

@jvmlet This is not the same issue, could you please create a new one?

@Mogztter , https://github.com/Mogztter/asciidoctor-kroki/issues/288

@Mogztter I'm interested in working on this issue but I don't know how to start it, for example Asciidoctor::PreprocessorReader#resolve_include_path doesn't seem to support a list of possible directories to look into, could you explain how the case should be handled? Thanks!

I'm interested in working on this issue but I don't know how to start it, for example Asciidoctor::PreprocessorReader#resolve_include_path doesn't seem to support a list of possible directories to look into, could you explain how the case should be handled? Thanks!

We should use our own mechanism. Asciidoctor::PreprocessorReader#resolve_include_path is used to resolve the include:: directive in Asciidoctor. The PlantUML !include directive achieves a similar goal but does not work exactly the same.
The goal here is to implement the following JavaScript function in Ruby:
https://github.com/Mogztter/asciidoctor-kroki/blob/6e6d86c6fca8a8a633eabf742b0fb8aac8b0d206/src/preprocess.js#L90


nimble-tab alert support

😯 Problem to Solve
Nimble tabs should support displaying a visual "alert" indication. This is needed by SL to indicate active/inactive alarms.
💁 Proposed Solution

Here are some more insights about implementation details from a meeting chat.
[Yesterday 17:38] Brandon O'Keefe
Just a heads up that the Nimble Components spec includes the placement and size of that red dot. In the development spec the state is tagged as Alert_'' ''
Screen Shot 2021-09-23 at 10.34.00 AM.png
Screen Shot 2021-09-23 at 10.37.47 AM.png
[Yesterday 17:44] Lukas LK. Kall
I didn't noticed that while implementing the tabs. We might want to add another property to the element so it's easier to enable the alert. Or even set the color of it 🤔
[Yesterday 17:45] Joshua Prewitt
we will need to set the color since we use it to indicate when there is an active alarm vs inactive alarm that just needs to be acknowledged
[Yesterday 17:46] Milan Raj
We can give it a color as defined by the design system and expose it as a CSS Part where the application consumer can then style however they need 👍
[Yesterday 17:46] Milan Raj
(hopefully with tokens from the design system to represent those different color states)
[Yesterday 17:49] Brandon O'Keefe
Joshua Prewitt
we will need to set the color since we use it to indicate when there is an active alarm vs inactive alarm that just needs to be acknowledged
Also, it will be important to have set the color since the specific red value is different between light and dark modes.

This feature is also being requested for DFFs in order to make the user aware of which tab (out of potentially many) contains an error.


resolves #631 let theme control which toc levels have dot leaders

add toc_dot_leader_levels key to theme (0-based)
default to using dot leaders for all levels
slightly optimize how dot leaders are managed

How do I apply this PR to a forked asciidoctor-pdf?

Did you figure it out?

yep, git remote add mojavelinux yourpersonalrepo etc

:+1:  The README needs to be clearer about how to do this.


[Question]Tutorial 3(Pytorch, JAX) Test accuracy for the model with sigmoid activation function

Thank you for your great tutorials!
I have a question about your comment on the test accuracy for the model with sigmoid activation function.
(It is under cell 17 for pytorch and under cell 18 for JAX)
You mentioned the result with sigmoid is very poor and, coincidentally, the model for JAX is trained, but is it because the model for pytorch is not trained well and is the result for JAX is correct?
I re-trained the model for pytorch and I found the training stops at epoch 8, because the result of epoch 1 is better than epoch 2-8.
This means the saved model is the result of epoch 1.
I changed "patient" variable from 7 to 50 and I got a similar result with JAX.
Thank you.

Hi, the sigmoid model is indeed a fun one to play around in this tutorial. :)
I had tried a couple of trainings in PyTorch with 50 epochs and noticed that some start learning suddenly, but many continued to fail even after longer trainings. You need to be a bit lucky that the gradients don't cancel each other out too much in the early layers and actually start learning. In JAX, the sigmoid networks tend to go slightly more stably to the learning regime. At the same time, when you optimize the initialization, add some normalization or use Adam, the MLP also trains relatively good with sigmoid activation functions. Nonetheless, the idea of the sigmoid training was to show that one shouldn't use sigmoid as the main hidden activation function in a network, since it brings several drawbacks. So I would recommend using other activation functions than trying to over-optimize the sigmoid network :)

Thank you for your comment.

I had tried a couple of trainings in PyTorch with 50 epochs and noticed that some start learning suddenly, but many continued to fail even after longer trainings.

I ran a code shown below and all test accuracies were higher than 75%...
Did many models with sigmoid really fail to learn?

for i in range(50):
print(f"Training BaseNetwork with {i} ")
set_seed(i)
act_fn = Sigmoid()
net_actfn = BaseNetwork(act_fn=act_fn).to(device)
train_model(net_actfn, f"FashionMNIST_sigmoid_{i}", overwrite=False, patience=50)

It is true the learning start suddenly.

So I would recommend using other activation functions than trying to over-optimize the sigmoid network :)

I agree this.
Thank you.


Remove extraneous white space

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve


Figure out why Python containers don't respond to SIGTERM

When run in a container, the Python applications built with these buildpacks don't respond to SIGTERM. When the same commands are run in a local execution environment, the applications respond correctly.
https://github.com/boson-project/buildpacks/pull/60#issuecomment-809603432

Seems that for some reason python is ignoring most of the signals when running with pid 1.

Well, at least default handlers are not working when pid == 1.
This could be fixed by installing own handlers:
def receive_signal(signal_number, frame):
    sys.stderr.write('Received: ' + str(signal_number) + "\n")
    sys.exit(1)
    return

signal.signal(signal.SIGTERM, receive_signal)
signal.signal(signal.SIGINT, receive_signal)

Note that code above terminates python quite ungracefully by sys.exit(), maybe it could be done better.

@lance I looked into it a bit ^^^

this probably should be fixed in parliament project

@matejvasek see https://github.com/boson-project/buildpacks/pull/70

this probably should be fixed in parliament project

After reading more about the problem, I don't think the solution belongs with parliament, since that would require that parliament actually implement a lot of stuff that is unrelated to what it does - e.g. reaping zombie and orphaned processes. Other tools like dumb-init are explicitly designed for this.

After reading more about the problem, I don't think the solution belongs with parliament, since that would require that parliament actually implement a lot of stuff that is unrelated to what it does - e.g. reaping zombie and orphaned processes.

I think it should be fixed in both:
https://github.com/boson-project/buildpacks/pull/68
https://github.com/boson-project/parliament/pull/2

implement a lot of stuff that is unrelated to what it does - e.g. reaping zombie and orphaned processes.

why?

why?

Well, that's what an init process is supposed to do. But of course, you are right to question this since parliament was previously PID 1. So, fair point. :)

But of course, you are right to question this since parliament was previously PID 1. So, fair point. :)

Actually it wasn't hence #68.

Well, that's what an init process is supposed to do.

I know this is true for standard OS, but I have no idea if that's needed in OCI where the process is the only one.

withoud exec:
docker top quirky_solomon
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
t1000               170975              170953              0                   15:59               pts/0               00:00:00            /bin/sh
t1000               171064              170975              10                  16:00               pts/0               00:00:00            python -m parliament

with exec:
docker top nifty_swanson                             SIGINT
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
t1000               174492              174471              22                  16:06               pts/0               00:00:00            python3 -m parliament .

I know this is true for standard OS, but I have no idea if that's needed in OCI where the process is the only one.

Although noting prevents user from calling fork, no idea what happens then.

@lance  I am not against dumb-init, but if it's the right solution we should also use it in another runtimes, right?

@lance I am not against dumb-init, but if it's the right solution we should also use it in another runtimes, right?

Probably... I don't know enough about what the other runtimes are doing to avoid this to say with any confidence though.

Rather than making that big change today (1.14 cutoff day) for all runtimes, I think we stick with your small changes like exec and the changes to parliament.


Backup issue when continuously inserting rows.

While doing insert queries to mysql cluster, i tried to take backup in parallel. For 2gb data backup size was 43MB and for 200GB data backup size was 33MB. And when i tried to restore the cluster from the backup file it got stuck with STATUS: Init:1/2.

Hi @goelsatyam, thanks for reporting this!
Can you please give me the logs of init container by running:
kubectl logs <restore-cluster-name>-mysql-0 -c clone-mysql

Also, I think will be useful the logs of xtrabackup tool from the pod from where the backup was taken:
kubectl logs <original-cluster-name>-mysql-<slave index> -c sidecar

Note: make sure to strip sensitive data from logs if the case.
Thanks in advance!

Logs of init container:-
Create rclone.conf file.
Create google-credentials.json file.
2019-03-15T09:34:48.604Z	INFO	sidecar	environment is not set	{"key": "MYSQL_REPLICATION_USER"}
2019-03-15T09:34:48.604Z	INFO	sidecar	environment is not set	{"key": "MYSQL_REPLICATION_PASSWORD"}
2019-03-15T09:34:48.604Z	INFO	sidecar	environment is not set	{"key": "MYSQL_METRICS_EXPORTER_USER"}
2019-03-15T09:34:48.604Z	INFO	sidecar	environment is not set	{"key": "MYSQL_METRICS_EXPORTER_PASSWORD"}
2019-03-15T09:34:48.604Z	INFO	sidecar	environment is not set	{"key": "MYSQL_ORC_TOPOLOGY_USER"}
2019-03-15T09:34:48.604Z	INFO	sidecar	environment is not set	{"key": "MYSQL_ORC_TOPOLOGY_PASSWORD"}
2019-03-15T09:34:48.604Z	INFO	sidecar	clonning command	{"host": "cluster-test-b-2-mysql-0"}
2019-03-15T09:34:48.614Z	WARN	sidecar	failed to obtain master from orchestrator, go for default master	{"master": "cluster-test-b-2-mysql-0.cluster-test-b-2-mysql-nodes.default"}
2019-03-15T09:34:48.614Z	INFO	sidecar	cloning from bucket	{"bucket": "s3:dataplatform-sql-backup/backup/cluster-test-2-2019-03-15T09:29:32.xbackup.gz"}
encryption: using gcrypt 1.7.6-beta
2019/03/15 09:34:48 DEBUG : rclone: Version "v1.46" starting with parameters ["rclone" "-vv" "--config=/etc/rclone.conf" "cat" "s3:dataplatform-sql-backup/backup/cluster-test-2-2019-03-15T09:29:32.xbackup.gz"]
2019/03/15 09:34:48 DEBUG : Using config file from "/etc/rclone.conf"
2019/03/15 09:34:49 DEBUG : pacer: Reducing sleep to 0s
xbstream: Can't create/write to file './xtrabackup_logfile' (Errcode: 17 - File exists)
xbstream: failed to create file.

I think that those logs are not from the first fail, those are the logs after a failed happened before and leave behind some files and it does not reflect the real issue. Maybe the operator should run a cleanup before doing a backup restore (but this is a different issue).

I think is problem is while taking the backup. Because when i am took the backup without doing any queries in parallel,  2gb data backup size was .35GB and for 200GB data backup size was 36GB. In both cases i was able to successfully restore the cluster from the backup file of both 2GB and 200GB.

That may be a problem of xtrabackup. I will try to reproduce it.
Do you still have the logs of the container where the backup was taken, to examine them?

logs of the container where the backup was taken:-
Create rclone.conf file.
Create google-credentials.json file.
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MY_CLUSTER_NAME"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MY_NAMESPACE"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MY_SERVICE_NAME"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "INIT_BUCKET_URI"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "ORCHESTRATOR_URI"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MYSQL_REPLICATION_USER"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MYSQL_REPLICATION_PASSWORD"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MYSQL_METRICS_EXPORTER_USER"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MYSQL_METRICS_EXPORTER_PASSWORD"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MYSQL_ORC_TOPOLOGY_USER"}
2019-03-15T11:49:01.566Z	INFO	sidecar	environment is not set	{"key": "MYSQL_ORC_TOPOLOGY_PASSWORD"}
2019-03-15T11:49:01.566Z	INFO	sidecar	take a backup	{"host": "cluster-test-2-mysql-1.cluster-test-2-mysql-nodes.default", "bucket": "s3://bucket-url/cluster-test-2-2019-03-15T11:48:50.xbackup.gz"}
2019-03-15T11:49:01.566Z	INFO	sidecar	initialize a backup	{"host": "cluster-test-2-mysql-1.cluster-test-2-mysql-nodes.default", "endpoint": "/xbackup"}

@AMecea The table in which i was inserting the rows didn't had any primary key.
can this cause the problem?

@AMecea  Getting the same  error when I tried it with table having primary key.

Do you have a script to reproduce this? I need to reproduce it to inspect closely the logs and to tweak some parameters to find a solution. Also, I want to write an e2e test on this case.

You can use this python script:-
import time
import sys
import mysql.connector
def insert():
sql = "INSERT INTO user (firstname, lastname, address) VALUES (%s, %s, %s)"
val = []
for i in range(100000):
val.append(("l_"+str(i), "f_"+str(i), "a_"+str(i)))
mydb = mysql.connector.connect(
    host="master_cluster_IP",
    user="root",
    passwd="root_password",
    database="test_data"
    )
mycursor = mydb.cursor()
mycursor.executemany(sql, val)
mydb.commit()

total_time = time.time()
for i in range(int(sys.argv[1])):
start_time = time.time()
insert()
print ("time taken for " + str(i*100000) + "-" + str(i+1)*100000 + " :" + str(time.time() - start_time))
run the script by:-  python  app.py 100
I ran this script from two different shells at same time.

Thanks for the script, I ran it and I was able to reproduce the error.
The logs that I was looking for are those:
xtrabackup: error: log block numbers mismatch:
xtrabackup: error: expected log block no. 567753, but got no. 1092033 from the log file.
xtrabackup: error: it looks like InnoDB log has wrapped around before xtrabackup could process all records due to either log copying being too slow, or  log files being too small.
xtrabackup: Error: xtrabackup_copy_logfile() failed.
2019-03-19T14:27:18.199Z	ERROR	sidecar	failed waiting for xtrabackup to finish	{"error": "exit status 1"}

Because the writes are too many and the innodb log files are too small the data from the log file is flushed before Xtraackup finishes copying files. What you can do is to increase the innodb_log_file_size (by default is computed based on requested memory (default 1G)).
This is a valid bug the backup should fail but instead succeeds.

@AMecea Where i can see the  xtrabackup logs?

The Xtrabackup logs are in the sidecar container of the node from where the backup was taken, and you can see them by running:
kubectl logs <original-cluster-name>-mysql-0 -c sidecar

If i got unexpected high number of insert requests and innodb_log_file_size  is small for the amount of load server is getting, then my backup will fail. Can we lock the slave pod so that atleast we can take the backup of the data already present at that time?

Yes, we can do that, we can issue a STOP SLAVE SQL_THREAD query before doing a backup. This logic can be under a feature flag on the MysqlClusterresource (like a backupMode field). But this is a different issue, so please open a new one. I will be happy to review and merge a PR that does that :smile:
PR #260 will fix the issue of having corrupt backups with success status.


Auto eval code strings

In my production server I keep needing to do:
def crud_handler():
    request = __import__('user_posted_json')
    eval(request.method)(db_connection, request.args)

Please consider allowing this syntax so my code looks cleaner:
def crud_handler():
    request = __import__('user_posted_json')
    request.method(db_connection, request.args)

I think adding a __call__ method on DottedDict that wraps eval should do it.

Ok, so I waited 20+ hours for a fix here and finally gave up on waiting and did it myself. Please merge #25 so I can close this long standing issue at last.


Refactor optimizer data structure from graph to tree

Now optimizer maintains a graph data structure to store plan nodes, but actually we just use graph as tree. We don't benefit from graph, instead it makes code harder to read.

Removed graph implementation, new implementation will refer to datafusion https://github.com/apache/arrow-datafusion/blob/main/datafusion/optimizer/src/optimizer.rs


git integration

i'd like to have a visual interface for git integrated in the IDE. I'd like to see a graph with all its' branches and would like to be able to click on the different commits to see what files were changed and when clicked on the file, to see the changes within them

Have you tried the GitLens extension?
I would recommend that you give that a try and it has a lot of git related functionality on top of what is built in to VS Code.

recommend : https://marketplace.visualstudio.com/items?itemName=mhutchie.git-graph

Have you tried the GitLens extension? I would recommend that you give that a try and it has a lot of git related functionality on top of what is built in to VS Code.

@lszomoru GitLens cramms everything into the Source Control side panel. Because GitLens is a third-party extension, it's trying to achieve what it can with the tools it is given, rather than having the freedom to consider how the best git GUI would look like if there were no contraints. I don't think it would look like an accordion with a bunch of items.

The feature request described here is covered by the new "Source Control Graph" view.


Added check for weak pin when creating one

This PR adds a check for weak PINs when the user wants to create / update one. After a user fills in the 6 digit PIN, it is checked against a list of known weak pins. This is done with an array of values, as a regex is not a good fit for the consecutive digits in increasing / decreasing order, so it is much simpler and easier to understand by defining the weak pins this way. After a weak PIN has been found, the user will be presented with an alert as shown below:

Then the user can decide whether they want to keep the current PIN. When they press 'No', the PIN will be cleared and they can try another. By choosing 'Yes', the user will simply be send to the confirmation screen. This way you don't pressure a user into changing the PIN, but you can give them a little hint that they should choose another one if they want their funds to be safe.

Nice work 👍


Select is not implementing {...$$restProps}

Hi!,
as per title, it seems that Select is not implementing {...$$restProps}, therefore, if I try to set a custom attribute, like a  data-testid, that won't be there on the <select> item as it applies for other components, like TextInput
Is this on purpose or is it just missing and will be possibly be fixed?
thanks

restProps is forwarded to the top-level element:
https://github.com/IBM/carbon-components-svelte/blob/master/src/Select/Select.svelte#L83

ok thanks.


#14010: fix caching issues with moreh_adam and moreh_adamw

Ticket
Closes #14010
Problem description
adam and adamw specified the step operation attribute that is provided at runtime as a non-cacheable argument. As such when the step changes the cache is always regenerated even though the cache item is valid if the argument is overriden accordingly
What's changed

Override compute_program_hash with custom hash implementation that does not use the step argument
Update override_runtime_arguments with new item for step

Checklist

[x] Post commit CI passes
[ ] Blackhole Post commit (if applicable)
[ ] Model regression CI testing passes (if applicable)
[ ] Device performance regression CI testing passes (if applicable)
[x] New/Existing tests provide coverage for changes

CI run: https://github.com/tenstorrent/tt-metal/actions/runs/11434570794

CI is OK: https://github.com/tenstorrent/tt-metal/actions/runs/11478694436/
I will proceed with merging


[Triton-MLIR][BACKEND] Add argmin / argmax implementation for ReduceOp

Hi @qliu93, I've introduced a new utility in the triton-mlir branch, please revise the code accordingly.


Log more fields, establish provenance for domains / services

Fixes (most of) these issues: fixes #253, fixes #226, fixes #144.
Logs:

rootDomain that each domain came from
what scan discovered each domain / service
who created a role
who approved a role
who created an organization
who created a scan

This PR does not log the following (and I think we probably don't need them for the initial launch):

which user opted in / out of scans, and at what time
updates of root domains for organizations

Reviewing within the next hour


如果文件的状态为“已删除”，应该怎么办呢？

一些在u盘中的重要文件，被检测出特洛伊木马，警告级别为严重，状态为已删除，并且标注“此威胁或应用已被允许，将来不会修正”，应该怎么办呢？这些文件比较重要。

文档详细信息
⚠ 请勿编辑此部分。 docs.microsoft.com ➟ GitHub 问题链接必须具有此部分。

ID: de416fda-04fb-6780-6675-020fdc2426ba
Version Independent ID: ede7f3c1-ba3a-e331-80d8-c992b3d7f9a4
Content: 在 Windows Defender AV 中还原隔离的文件
Content Source: windows/security/threat-protection/windows-defender-antivirus/restore-quarantined-files-windows-defender-antivirus.md
Product: w10
Technology: windows
GitHub Login: @Dansimp
Microsoft Alias: dansimp

如果出现病毒/木马/恶意软件感染，则应在本地寻求帮助。 确保仅在具有良好防病毒软件的干净，隔离的计算机上访问这些文件，而不必担心会传播感染。
(In case of a virus / Trojan / malware infection, you should seek help locally. Make sure to only access those files on an clean, isolated computer with good anti-virus software where you do not risk spreading the infection.)

Thank you! But it had told me that the files had been deleted, can i still find them after establishing a secure environment?

Not sure if they are 100% deleted or if they are locked inside some vault or hidden files. If you want to try to restore files, you should definitely shut down the computer in question, before any more changes are done to it, then ask for help for someone to perform data retrieval on either the hard drive or the entire computer (depending on type of device), using a separate OS (Linux or something else than the original Windows OS) and also have access to repair, scan and file recovery tools for your situation. This is very much a hands-on operation, so you need help from local expertise.

All right, it sounds not easy to do. Thank you!

Correct, the file data and drive(s) in question should be treated with respect and great care. I can only wish you good luck in recovering your data.

I will and I'll close the question, thank u again.


Meta: supporting S3 compatible APIs?

Right now we test against Minio and Ceph for non-AWS implementations of S3. We've found issues where Rusoto works with AWS S3 but not third party services. Example: https://github.com/rusoto/rusoto/issues/1463 .
Questions:

should we support these non-AWS services?
if we choose to support them, how much effort should Rusoto maintainers put forth for them?

My opinion: due to time constraints for the entirely volunteer effort backing Rusoto, maintainers should treat these non-AWS services as a low priority. I'm comfortable with accepting PRs around making Rusoto work with them but not actively spending time improving non-AWS services.
Feedback and discussion welcome! 😄

IMHO, I would prefer a robust AWS-centric Rusoto that works well with Amazon at the expense of not being so compatible with third parties than a half baked, "please everyone" library that does not work as well.
Focus is important to attain quality, I think.

I added info to the root README in https://github.com/rusoto/rusoto/pull/1518 . 👍


Payment issue for E/App Issue #15595

I created this issue to process payment for $1000 PR review flat rate to @thesahindia
The contributor reviewed this PR - https://github.com/Expensify/App/issues/15595

https://www.upwork.com/ab/applicants/1637911004499845120/offers

Thanks for working on this @slafortune!

Paid!


Release: @roots/bud ^2.0.0

The merging of this PR will mark the initial release of @roots/bud, now the name of the Roots webpack builder package.

This release will be tagged: 2.0.0.
This repo will be renamed: roots/bud.
The roots/bud repo will be renamed: roots/clover.
The PHP plugin support code (src/lib in the master branch) will need to be moved to another repo.

https://github.com/roots/bud-support/blob/3984f81c14b2b5438090e938f3a74a3034a988bf/src/bud/plugin/adapters/webpack/hotModuleReplacement.ts#L5
need to look for the remaining bud.options, bud.features, bud.paths references and update them to use the new  namespacing: bud.state.options, bud.state.features, bud.state.paths, etc.

From #5:

this might introduce new issues but is generally a big improvement.

@QWp6t these new additions should hopefully fix the problems you were having with yarn aliases.
https://github.com/roots/bud-support/blob/ac68b0baf2f6bd15a9fc50f5aa8ce47071a464ec/src/bud/api/compile.ts
by attaching the compiler directly to bud it means we don't have to bounce the config back and forth between the bud module dir and the project dir. it also means that bud should resolve deps more consistently in different build environments.

Relevant docs PR https://github.com/roots/docs/pull/216
Documented a bunch of the more crucial functions, and included a couple higher level guides, as well.


Add tests for delete non-linked node, and add exitint linked node

for https://openscience.atlassian.net/browse/OSF-5579
Tests follow JSON api specs:
POST:
If all of the specified resources can be added to, or are already present in, the relationship then the server MUST return a successful response.
DELETE:
If the client makes a DELETE request to a URL from a relationship link the server MUST delete the specified members from the relationship or return a 403 Forbidden response. If all of the specified resources are able to be removed from, or are already missing from, the relationship then the serverMUST return a successful response.


Stuck at loading

Hello, when I try to play the game for WARHW at http://zorbio.openshiftdemos.com/, the game is stuck at loading. No matter how much time I wait (10+ mins).
Is that a problem with the server or the game (too big)? Here is the screenshot.

Thanks for resolving the issue / telling me it has nothing to do with development :D (Don't know who else to contact)

Hey @Wecros, thanks for the bug report.  It would be helpful to see if there are any errors in the browser console.  The Zorbio instance for WARHW is gone, but you can try to play at https://zorb.io and let us know if you have the problem there too.  If you do, seeing any console errors would be great.  Thanks!

zorb.io-1604575544515.log
Here is the log from the console.

Ahh, I think the region picker is trying to point you at a server that doesn't exist anymore.  @Jared-Sprague what do you think?

@Wecros This should be fixed now that Zorbio has moved to it's permanent home at OpenShift dedicated @ https://zorb.io
Let us know if you have any problems loading again! Thanks for the report! Closing for now.

Confirming that it has been fixed, fun game :)


Shortcode content rendered inside markdown list under certain circumstances

What version of Hugo are you using (hugo version)?

hugo v0.83.1-5AFE0A57+extended windows/amd64 BuildDate=2021-05-02T14:38:05Z VendorInfo=gohugoio

Does this issue reproduce with the latest release?
Yes
To reproduce you can use the following code. I'm not entirely sure what triggers this. If I do 2 CR/LF it works correctly. So please advice if i'm doing something wrong or if this may be a bug in the markdown processor  - or maybe related to some windows/linux cr/lf problem. Thanks
shortcode: specialp.html

{{index .Page.Params.special}}

test markdown
---
special: "my special text"
---
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
- xxxxxxxxxxxxxx
- xxxxxxxxxxxxxx
- xxxxxxxxxxxxxx
- xxxxxxxxxxxxxx
- xxxxxxxxxxxxxx
- xxxxxxxxxxxxxx

{{<specialp >}}

Discourse - that is a note you should have seen when opening this bug report that is not a bug.
index works differently from what you are doing there. It's index collection indexid. So it would be index .Page.Params "special". But as I wrote: Open your SUPPORT REQUESTS at the place the links when you opened this issue told you to.

Sorry for posting. Won't do again. I was convinced this was some bug. By the way it was completely unrelated to the wrong usage of index, but was caused by an invisible ZERO-WIDTH-SPACE between the CR LF. So after removing it , it is working


drivers: display: uc81xx: convert to MIPI DBI API

Convert UC81XX display to use MIPI DBI API, as this display uses a SPI
3/4 wire bus. In tree shields using this driver have also had their
devicetrees updated to use the new MIPI DBI SPI driver

@DineshDK03 or @andysan, could you test this PR? I do not have hardware to verify it, but would like to move this display to use the MIPI DBI interface (as it uses 3/4 wire SPI mode to communicate with an MCU, and the MIPI DBI class abstracts hardware IP that implements 3/4 wire SPI mode, as well as providing an emulated 3/4 wire SPI mode driver that uses the SPI interface)

Rebased to clear conflicts

Rebased to clear conflicts- note #73686 also touches the display driver build all test, so if either merges it will cause conflicts for the other. Not sure there's a better way to do this than keeping both updated though.


JSON serialization error on Fetch

Hi,
I noticed JSON format output is invalid for "la" field,
=> "la":0" is not valid JSON
=> "la":"0" should be the valid output
I needed to mask some labels for confientiality
Sample data :
 {"c":"xxx","l":{"sens":"Retour".app":"io.warp10.bootstrap","num_arret":"1663","id_type_titre":"1","numero_de_la_ligne":"207"},"a":{},"i":"108130483266709504,"la":0","v":[[1539583200000,-1.6705980012193322,48.1042109336704,4]]},{"c":"xxx","l":{"sens":"Aller",".app":"io.warp10.bootstrap","num_arret":"1663","id_type_titre":"1","numero_de_la_ligne":"41"},"a":{},"i":"284291214830745600,"la":0","v":[[1539667800000,-1.6705980012193322,48.1042109336704,2]]}

This should already be solved in HEAD on master

Do you confirm that the solution to #442 solves your issue?

Indeed fixed on HEAD


Example to generate dating app profiles- combines prompt templating with JSON generation

Let me know if there's any additional information you'd like me to add.
I figure this practical example will be useful to demonstrate more complex JSON generation alongside prompt templating (which is more useful the more complex the JSON structure becomes).

Awesome example! I noticed a comment mentioning that constr makes the compilation slow. Would you mind opening an issue so we track this?#272 should help with this.

@rlouf Thanks! I went ahead and ran the pre-commit hook and it passes now. One note on that which I also made in a comment in the code: mypy seems to have some outstanding issues with constrained pydantic typing. See the issue here:  https://github.com/pydantic/pydantic/issues/975
Unfortunately, that means in order to pass the pre-commit hook I had to add a type: ignore comment in my code.
As a sanity check, I also ran the pre-commit hook on the example in the README but unfortunately this failed the pre-commit hook for the same reason and requires a type: ignore comment to pass.
I'll also create an issue soon constr increasing runtime issue and reference this PR.

Thanks @brandonwillard! I'm a big fan of your guys' work on this, and see a lot of potential.
I wanted to try JSON generation with a more complex JSON that would benefit from doing joint generation of multiple fields. Dating profiles (relevant to my day job) seemed to fit the bill.


llvm-lit for tests?

Currently, the turbo loop pass is not tested. Instead, I run a few examples and check that things look right. This is slow and error prone.
https://llvm.org/docs/CommandGuide/lit.html

I could have this wrong, but reading through https://llvm.org/docs/WritingAnLLVMNewPMPass.html, it seems like using lit to test new passes requires registering the pass and building opt.
We could start off by just using FileCheck which I think would let us avoid building opt, which took a while locally.
Maybe it would be good to add a usage section to the readme, since I'm realizing now that I don't really know how to run the Turbo pass besides just copying the source into llvm/lib/Transforms and rebuilding opt. Is this the right way to start?
I know you showed me a couple examples of the pass IIRC

We could start off by just using FileCheck which I think would let us avoid building opt, which took a while locally.

Yeah, ideally we don't have to build opt but can just load the pass as a plugin.

Maybe it would be good to add a usage section to the readme, since I'm realizing now that I don't really know how to run the Turbo pass besides just copying the source into llvm/lib/Transforms and rebuilding opt. Is this the right way to start?
I know you showed me a couple examples of the pass IIRC

Yeah, that'd also be good. Currently, I'm doing something like:
opt -mcpu=native --load-pass-plugin=$LoopModelsBuildPath/libTurboLoop.so --passes=turbo-loop --disable-output $path/to/file.ll

Run meson compile or meson compile TurboLoop to build libTurboLoop.so. Note that it'll be .so on all operating systems, because it isn't a shared object (.so on Linux, .dylib on Mac, and .dll on Windows), but a plugin, and those are always .so.
Anyway, build the ".so", and then you can opt --load-pass-plugin it without needing to build your own opt.

Yeah, looks like FileCheck is what we'd have to use.
We're not actually producing code yet.
Is there a way to test optimization remarks?
If so, we could start converting print statements into optimization remarks, which we'd want to do anyway at some point.

Nice this is very useful

Is there a way to test optimization remarks?

Ref https://llvm.org/docs/Remarks.html I'll look into this.

Where can I find some good examples to test the pass on?
I'm guessing eventually we'll also want to run the pass on the test-suite


no-consecutive-blank-lines: Allow padding before headings

This is a direct extension of #115 (feel free to close it if this one is more relevant).
It is a common stylistic practice to leave padding before headings (or "at the end of sections", depending on how you see it).
I would like to propose an option for the no-consecutive-blank-lines to accept this stylistic choice (using ipsum for a more realistic content):
<!-- Beginning of the file, with empty first line -->

# Main Title

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec non ipsum vitae augue vehicula tincidunt tincidunt ut sapien. Vestibulum felis nisi, egestas quis ultrices at, gravida at ex.
Sed vitae tristique lectus. Nulla sodales neque ut tortor blandit molestie. Mauris interdum vulputate feugiat. Proin eleifend dignissim pulvinar. Nam elementum dapibus placerat.
Vivamus commodo ex quis blandit rutrum. Nunc euismod volutpat tortor, nec pharetra urna ultricies in.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec non ipsum vitae augue vehicula tincidunt tincidunt ut sapien. Vestibulum felis nisi, egestas quis ultrices at, gravida at ex.
Sed vitae tristique lectus. Nulla sodales neque ut tortor blandit molestie. Mauris interdum vulputate feugiat. Proin eleifend dignissim pulvinar. Nam elementum dapibus placerat.
Vivamus commodo ex quis blandit rutrum. Nunc euismod volutpat tortor, nec pharetra urna ultricies in.

## Section A

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec non ipsum vitae augue vehicula tincidunt tincidunt ut sapien. Vestibulum felis nisi, egestas quis ultrices at, gravida at ex.
Sed vitae tristique lectus. Nulla sodales neque ut tortor blandit molestie. Mauris interdum vulputate feugiat. Proin eleifend dignissim pulvinar. Nam elementum dapibus placerat.
Vivamus commodo ex quis blandit rutrum. Nunc euismod volutpat tortor, nec pharetra urna ultricies in.

## Section B

[...]

This helps visualizing sections tremendously and makes markdown files much prettier in my opinion.
Consecutive lines that appear in other places than before headings should still trigger warnings of course. What do you think?

If we do that, we should split it. Maybe like this:

Extra spacing between heading and preceding content (similar to remark-heading-gap, but in lint form, possibly an extension of no-consecutive-blank);
no-initial-lines (separate from no-consecutive-blank, so it can be turned off);
no-final-lines (separate from no-consecutive-blank, so it can be turned off).

I'm not sure I understand no-initial-lines and no-final-lines, could you elaborate about what would these 2 do?

I propose:

no-consecutive-blank-lines, to check spacing between nodes. It could have a an option for extra spacing before headings;
no-initial-blank-lines, to check that there’s no spacing before the first node (separate so it can be
controlled separately);
no-final-blank-lines, to check that there’s no spacing after the last node (separate so it can be controlled separately).

Does that make it clearer?

Yes it does, thanks!

I think this would be a good addition as an external rule! If anyone would like to take this up, I’d be able to help, but unfortunately I’m not able to do this myself.
❤

Whoops, sorry! I forgot about GH-115.
Are you still interested in helping?

I'm also quite overwhelmed these days, so I can't help unfortunately. Feel free to close my issues :)

Cheers @verekia!


Crash when using the cut and paste gadget

crash-2023-09-09_16.38.58-client.txt

The cause of this crash is Rubidium - please remove it and see if you still crash.


Refine deprecation mechanism

Context
Please see https://github.com/gradle/gradle/issues/3228 for context.

Thanks @bmuschko !


How do you set sails.config.beforeShutdown for all environments?

Is there a way of setting a root-level config value across all environments?
Specifically I am trying to set sails.config.beforeShutdown.
I have tried:

setting it separately in every config/env/*.js file.  This works, but is annoying to have to do for every environment, and potentially every new environment.
setting it in config/beforeShutdown.js.  This throws an error on sails startup:

error: Error: When using `aggregate`, modules must export dictionaries.  But module (`beforeShutdown`) is invalid:() => console.log('bye bye')

setting it in app.js's sails.lift() call.  This doesn't throw an error, but neither is the listener called.  I suspect this is a different kind of config:

const configOverride = rc('sails');
console.log('configOverride:', configOverride);
configOverride.beforeShutdown = () => console.log('bye bye 👋');

// Start server
sails.lift(configOverride, err => {
  if(err) {
    console.error('Failed to lift app:', err);
  }
});

Is there another option for setting this?

Hi, @alxndrsn!
We were able to override the default config using your second method and without incurring the error you got, although we did run into other issues. The problem is that this isn't really exposed functionality; you could probably work something out, but it wouldn't be within the officially supported/recommended Sails usage.
Feel free to share any workarounds you come up with, though, and good luck!

@madisonhicks thanks for the response.  You say:

We were able to override the default config using your second method and without incurring the error you got

I've tried again with a clean project on sails 1.2.3, but with the same result.  Please see example project at https://github.com/alxndrsn/sails-before-shutdown-via-config.
What am I doing wrong?

Thanks for the example, we'll checked it out.  By the way, what is the use case?
Appreciate all the help so far 👍

What is your use case for the beforeShutdown config?

Shutting down PostgreSQL connection pools cleanly.

Hey, @alxndrsn—
Sorry for the delay on this! Sails doesn't infer the type of config from the file name, so when modifying config files, you have to be specific. The format ends up being something like module.exports.configuredThing = () => console.log('some text');.

@madisonhicks Thanks - I think I see what you're saying.  I've updated the example app and it looks like it works now.  Is this change the fix you'd expect https://github.com/alxndrsn/sails-before-shutdown-via-config/commit/d9045ddbb341957f5a07d3613fe644b1c315d305?

@alxndrsn, it sure is!


Change edit text height

Is it possible to change the height of the search edit text?
Can you add a XML property or a way to change it programmatically?
Thanks

As explained in the README, you can probably use theme attrbitutes for that:
<!-- your custom theme -->
<style name="CustomFloatingSearchViewTheme" parent="AppTheme">
    <item name="editTextStyle">@style/searchViewEditTextStyle</item>
</style>

<!-- your custom search view text style with minimum height -->
<style name="searchViewEditTextStyle" parent="Widget.AppCompat.EditText">
    <item name="android:minHeight">128dp</item>
</style>

Then, apply your custom theme through app:theme:
<com.mypopsy.widget.FloatingSearchView
    ...
    app:theme="@style/CustomFloatingSearchViewTheme"
/>


Check that list of device configuration has at least one element

Check that list of device configuration has at least one element before inspecting first element

Thanks for the pull request. Unfortunately there is a syntax error due to the missing if keyword.

Sorry about that. There was a copy-paste error from my local copy. But I'm sure you get the gist.

Sure. I just commited it in https://github.com/SySS-Research/azurenum/commit/a00422a3905694093a9e8dddb59a45216b3faa39
Thanks!

Super - thank you!


Backport changes from Fast Sync to dev

This ports the changes in the block hashing scheme, storage databases and contract runtime from the feat-fast-sync branch to dev.
Some of the original PRs: #1563, #1649, #1754, #1837
Closes #1545

bors try

bors try

bors merge


chore: use common semantic PR and commit message checks

After this is merged, the old semantic checker can be disabled, and its config file removed from the repository.
This semantic PR title and commit message checker is intended to be common to all InfluxData repositories.
https://github.com/influxdata/validate-semantic-github-messages/
Checks are according to https://www.conventionalcommits.org/en/v1.0.0/

[x] Well-formatted commit messages
[x] Rebased/mergeable
[x] Tests pass

FYI I will follow up with removing the old semantic checker after merge.

Checks this regex - https://github.com/influxdata/validate-semantic-github-messages/blob/main/semantic_pattern.txt
LGTM


0.7.288 something broken with apero_astrometrics.py and new targets

Hi Neil
We have just completed the observations of an discretionary program and I wanted to process the data asap with the 0.7.288.
I set SCIENCE_TARGETS =nameofnewobj  in my ini.file but get an error message from Apero :
06:03:06.613-  |PROC|    processing recipe 'apero_fit_tellu_spirou.py' (FTFIT1)
06:03:07.902-!!|PROC| No objects found in astrometric database.
06:03:07.903-!!|PROC|   Please add objects to the astrometric database.
06:03:07.903-!!|PROC| Listname="SCIENCE_TARGETS"
06:03:07.903-!!|PROC|   (core.core.drs_database.py.DatabaseManager.find_objnames())
06:03:07.904-!!|PROC|   Objnames: "nameofnewobj"
06:03:08.181-|PROC| ***************************************************************************
06:03:08.193-@!|PROC| W[40-003-00005]: Recipe apero_processing has NOT been successfully completed
06:03:08.203-|PROC| ***************************************************************************
I added the target in the obj db (although it's not relevant for this H>12 star) but still get the error message.
The weird thing is that if I set SCIENCE_TARGETS = All,   it works, no error message.   But I don't want (can't yet) to process all stars, only that new single object for the moment.  I don't understand why Apero is behaving this way.
L

So the astrometric database is used for more than just the pm now, we rely on science targets being there for multiple sets including the template creation (mostly for the name).
Its a weekend so I'm not near my computer but I guess nameofnewobj is not your target name and you redacted this information from github? If so I assume your object is the last one in the list (a TOI object) I see it looks like you entered a Gaia DR2 id as the input name- I would suggest you check exactly what the name is in the header (you could use precheck to tell you what apero is looking for - it will tell you the name.... maybe you need to add whitespace between i.e. TOI XXXXXYYYYY I see that only TOI-XXXXXYYYYY is in the aliase list, I always recommend adding a few alaises - that might work (and precheck will tell you when infact it is in the database.
If you confirm with precheck that it is in the database then we might have a bug. Again its a weekend I'm not going to debug it now. But if you really want data now you never HAVE to use apero processing you can do the steps manually (apero_processing is just a convience).
i.e. you could do the following commands:
apero_preprocess_spirou.py {night} {filename}

apero_extract_spirou.py {night} {filename}

apero_fit_tellu_spirou.py {night} {filename}

If you are unsure which commands to run you can run apero_processing.py for another object for one night in test mode --test=True
and it will print a list of commands that will be run - you can easily run these one by one for your filename(s) or add them to a run.in file as follows:
This is the full run.ini file

# Note this is a example file
#   Please copy this before making changes
# FOR USE WITH VERSION 0.7.288 (2023-09-06)

# -----------------------------------------------------------------------------
# Core variables
# -----------------------------------------------------------------------------
# Define run name
RUN_NAME = My custom commands

# Define whether to send email (required yagmal)
#   to install yagmail: pip install yagmail
# Note this currently is untested
SEND_EMAIL = False

# Define email address to send/recieve (send to self)
# Note this currently is untested
EMAIL_ADDRESS = None

# Define single observation directory (for all nights to "All")
RUN_OBS_DIR = All

# Define observation directories to exclude ("None" means no filter)
#    EXCLUDE_OBS_DIRS = 2018-07-31, 2018-08-05, 2018-12-18, 2018-12-19, 2019-01-17
EXCLUDE_OBS_DIRS = All
# Define observation directories to include ("All" means no filter)
#    INCLUDE_OBS_DIRS = 2018-07-31, 2018-08-01, 2018-08-02, 2018-08-03
INCLUDE_OBS_DIRS = All

# Define a pi name list for filtering by ("All" means no filter)
PI_NAMES = All

# define reference observation directory
REF_OBS_DIR = 2020-08-31

# Number of cores (if negative uses N-ABS(CORES), if zero uses N-1 cores)
CORES = 5

# Stop on exception
STOP_AT_EXCEPTION = False

# Run in test mode (does not run codes)
TEST_RUN = False

# Whether to process engineering data
USE_ENGINEERING = False

# Whether this is a trigger run
TRIGGER_RUN = False

# Use odometer reject list
USE_REJECTLIST = True

# Recalculate templates if template already exists
RECAL_TEMPLATES = True

# Update object database from googlesheet - only recommended if doing a
#    full reprocess with all data (will cause inconsistencies if googlesheet
#    has been updated and you are not reprocessing all new data)
UPDATE_OBJ_DATABASE = False

# Update reject database from googlesheet
UPDATE_REJECT_DATABASE = True

# Update the index database. WARNING only do this if precheck or a previous
#   apero_processing has been run. This will result in bad things if the
#   index database is not update to date.
UPDATE_FILEINDEX_DATABASE = True

# Select which databases to update. WARNING. You must be 100% sure that
#   databases left out of this list are up to date, otherwise APERO will
#   fail in bad ways. Options are "All", or any combination of the following:
#   "raw", "tmp', "red", "out"
UPDATE_IDATABASE_NAMES = All

# -----------------------------------------------------------------------------
# Run information
# -----------------------------------------------------------------------------
#
#   Format:
#
#   id##### = command
#
#   Must start with "id" followed by a number
#     The number identifies the order to process in
#
#   If command is a sequence will process all in defined sequence
#      valid sequences are:
#           pp_seq, pp_seq_opt, full_seq,
#           limited_seq, ref_seq, calib_seq,
#           tellu_seq, science_seq, eng_seq

id00000 = apero_preprocess_spirou.py {night} {filename}

id00001 = apero_extract_spirou.py {night} {filename}

id00002 = apero_fit_tellu_spirou.py {night} {filename}

you can then run this through apero_processing - it wont use a sequence therefore you shouldn't have the problem above.
Obviously update the stuff above "Run information" with the default stuff you use from another run.ini  - but you do not need any RUN_XXXX or SKIP_XXXX stuff (as no sequences are being used) apero should just run your commands in parallel where possible (i.e. it will run all your preprocess if the id's are in sequential order)

It may also have to do with this, we just got an error updating the object database:
20:14:04.342-**|DatabaseError|E[00-002-00047] Pandas.to_sql <class 'sqlalchemy.exc.DataError'>: (mysql.connector.errors.DataError) 1406 (22001): Data too long for column 'TEFF_SOURCE' at row 1380

It seems your TEFF_SOURCE was too long (it needs to fit in a header so < 80 characters), I've edited the row.
If you weren't getting this error it means your astrometric database is not up-to-date, so another fix may be to make sure the following is True in your run.ini file:
# Update object database from googlesheet - only recommended if doing a
#    full reprocess with all data (will cause inconsistencies if googlesheet
#    has been updated and you are not reprocessing all new data)
UPDATE_OBJ_DATABASE = True

If it isn't you need to update your astrometric database yourself i.e. with:
apero_database.py --update --dbkind='astrom'

In general we keep the database "offline" in case mistakes are made - so you do need to update it if you add objects!

Thanks Neil !   the UPDATE_OBJ_DATABASE = True solved the issue.
Sorry for the nameofnewobj  I was not sure this ticket on github was public or not.
it's good to know I can have lines like id00000 = apero_preprocess_spirou.py {night} {filename}  in an ini file.

It is public so yes probably best to leave the name out as you did!


System.TypeLoadException: Method 'NormalizeName' after upgrading to 1.0.4

I have a project where I was running 1.0.2. And it was working fine, but after upgrading to 1.0.4 I get this error:
Unhandled Exception: System.TypeLoadException: Method 'NormalizeName' in type 'Amazon.AspNetCore.Identity.Cognito.CognitoKeyNormalizer' from assembly 'Amazon.AspNetCore.Identity.Cognito, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=885c28607f98e604' does not have an implementation.
   at Microsoft.Extensions.DependencyInjection.CognitoServiceCollectionExtensions.InjectCognitoUser[TUser](IServiceCollection services, Action`1 identityOptions)
   at Microsoft.Extensions.DependencyInjection.CognitoServiceCollectionExtensions.AddCognitoIdentity(IServiceCollection services, Action`1 identityOptions, String prefix)
   at Server.configureServices(IServiceCollection services) in C:\Users\marnee\Documents\mister\application\mister-portal-cognitoupdate\src\Mister.WebHost\Program.fs:line 98
   at Server.app@167-3.Invoke(IServiceCollection services) in C:\Users\marnee\Documents\mister\application\mister-portal-cognitoupdate\src\Mister.WebHost\Program.fs:line 167
   at Saturn.Application.Run@77-2.Invoke(FSharpFunc`2 fn) in D:\Programowanie\Projekty\Saturn\Saturn\src\Saturn\Application.fs:line 77
   at Microsoft.FSharp.Primitives.Basics.List.iter[T](FSharpFunc`2 f, FSharpList`1 x)
   at Saturn.Application.Run@75-1.Invoke(IServiceCollection svcs) in D:\Programowanie\Projekty\Saturn\Saturn\src\Saturn\Application.fs:line 77
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at Saturn.Application.run(IWebHostBuilder app) in D:\Programowanie\Projekty\Saturn\Saturn\src\Saturn\Application.fs:line 570
   at Server.main(String[] _arg1) in C:\Users\marnee\Documents\mister\application\mister-portal-cognitoupdate\src\Mister.WebHost\Program.fs:line 177

I am running on .Net core runtime version 2.2.105
Is the latest version not compatible with this .Net core version? There was a fix to the way it handled password requirements, and I need this in my project, but I am not ready to upgrade to 3.0. Any suggestions?

I'm closing this for now.  Feel free to reopen it if you can provide a reproduction solution.


Expose ToQueryable in order to use along with OData (ApplyTo)

Hi I made this PR because I want to combine the query it is built with MultipleResultQuery with another IQueryable provider like OData, the repository was not capable to expose a IQueryable, so the query can't be extended.
This is what I can do now:
public async Task<IActionResult> SearchDto(ODataQueryOptions<Entity> oquery)
        {
            var repo = _unitOfWork.Repository<Entity>();
            var query = repo.MultipleResultQuery()
                                .Include(source => source.Include(doc => doc.AnotherEntity))
                                .OrderByDescending(s => s.CreationTime);

            var queryable = repo.ToQueryable(query);  // Obtain an IQueryable

            var entities = oquery.ApplyTo(queryable);          // Apply OData filters to the IQueryable

            return Ok(entities);
        }

I also bump nuget version to 6.0.7
and update few dependencies to the latest compatible version.

Hi!
Thanks for the contribution!
I'd appreciate If you could do the same for the 2.x, 3.x and 5.x implementations.
Fell free to submit other PRs.
Everything from my side is done, we can go ahead with this PR and merge it.


FEATURE - Enhance Fiori freestyle API for integration into easy-ui5

As a developer,
I want to generate Fiori freestyle project with more flexibility,
so that I can choose my own view names or select the source of UI5
Description
When @vobu tried integrating the fiori-freestyle template into the easy-ui5 following features were not possible to be set:

Configure a custom view/controller name instead of the currently hardcoded name
Configure the target url that is used to load UI5 in the preview
Generate a project using OpenUI5 instead of SAPUI5.

Technical Design
Coming up with a technical design that works with both @sap/generator-fiori as well as with easy-ui5 needs to be evaluated.
Acceptance Criteria
GivenWhenThen format (https://martinfowler.com/bliki/GivenWhenThen.html)
Given I am using easy-ui5
when I execute the newly integrated fiori-freestyle templates
then I will be able to define my custom view name.
Given I am using easy-ui5
when I execute the newly integrated fiori-freestyle templates
then I will be able to use OpenUI5 loaded from a custom location.
Given I am using @sap/generator-fiori
when I generate a Fiori freestyle project
then I want it to behave as today without requiring me to enter more information.
Notes
none
Tasks

[x] Discuss an extension of the API with experts from both @sap/generator-fiori as well as with easy-ui5
[x] Implement the agreed API extension
[x] Experts from both @sap/generator-fiori as well as with easy-ui5 verify the implementation
[x] Unit Tests added
[x] Test Cases defined

@vobu could you please check if I collected all requested changes from #65?

@vobu could you please check if I collected all requested changes from #65?

in principle all good.
would still word things differently for "Description" > "2."
"Configure a custom bootstrap URL for consuming UI5 runtime resources"
this would also make "3." obsolete IMO.
Acceptance criteria are spot on.

The features described are implemented and the acceptance criteria is fulfilled.
@vobu did we miss something?

just FTR: /me thinks, we're good 👍


Update the game over message

Update the game over message so people know how to play again!

Complete


When submitting job through kubectl(k8s), it doesn't execute properly.

Hi,
I refered the "README for IBM Spectrum Computing LSF-Kubernetes Integration, Tech preview" but also the knowledge center guide of LSF Connector for Kubernetes and have tested the same environment.
After all configuration, we've encountered some issues of job submitting. When I submit the job through LSF, it work properly. However, if job is submitted through the Kubernetes, it shows pending/error state and K8S pod terminated. We tried allocate both cpu and gpu resources but doesn't work either.
Here's the each versions of docker, nvidia-docker, ICP for k8s distribution. Also I attached some logs.
Is it bug ? Or can I assume any configuration missing ?
[root@lsfsuite log]# nvidia-docker version
NVIDIA Docker: 2.0.3
Client:
Version:           18.09.3
API version:       1.39
Go version:        go1.10.8
Git commit:        774a1f4
Built:             Thu Feb 28 06:33:21 2019
OS/Arch:           linux/amd64
Experimental:      false
Server: Docker Engine - Community
Engine:
Version:          18.09.3
API version:      1.39 (minimum version 1.12)
Go version:       go1.10.8
Git commit:       774a1f4
Built:            Thu Feb 28 06:02:24 2019
OS/Arch:          linux/amd64
Experimental:     false
[root@lsfsuite log]# docker -v
Docker version 18.09.3, build 774a1f4
[root@lsfsuite log]# nvidia-docker -v
Docker version 18.09.3, build 774a1f4
[root@lsfsuite log]# kubectl describe daemonset platform-ui -n kube-system |grep  ICP_VERSION
ICP_VERSION:                        3.1.0
[root@lsfsuite log]# kubectl get nodes
NAME         STATUS    ROLES                                 AGE       VERSION
<IP_ADDRESS>   Ready     etcd,management,master,proxy,worker   39d       v1.11.1+icp-ee
These are all logs from batch-drive.log. There's nothing inside kubebridge.lsfmaster.log
[lsfadmin@lsfsuite log]$ cat batch-driver.log
Log file created at: 2019/11/27 14:44:12
Running on machine: lsfsuite
Binary: Built with gc go1.11.2 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
W1127 14:44:12.720253   30795 authorization.go:47] Authorization is disabled
W1127 14:44:12.720353   30795 authentication.go:55] Authentication is disabled
I1127 14:44:12.720360   30795 deprecated_insecure_serving.go:49] Serving healthz insecurely on [::]:10501

Hi @brlee08
What version of ICP are you using?  There are some docs here for using GPUs in ICP:  https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_applications/deploy_app_gpu.html  You can change the ICP version of the doc to suit your ICP version.  If you are using the OpenShift based version the instructions are a bit different.  That version does not use the nvidia-docker, but it does use the nvidia-device-plugin.  Instructions for that version are here:  https://github.com/zvonkok/origin-ci-gpu/blob/release-3.11/doc/How to use GPUs with DevicePlugin in OpenShift 3.11 .pdf
Mark

Thanks Mark. ICP version I'm using is 3.1.0. I'll try upgrading the version, and test again.

We will be coming out with a refreshed tech preview soon.  Can't say when yet.  It will be more robust and the parallel / elastic job support is improved.
If you are using LSF, you can download Service Pack 9 and get all the bits now.

Closing this as it's really old


Versatile Thermostat does not follow target temperature from underlying thermostat

Is your feature request related to a problem? Please describe.
i’m using (tying to) use versatile thermostat to add features to my locally controlled tado thermostats. the underlying thermostats are integrated as homekit devices. if i manually change the (target) temperature at the physical thermostat, the homekit devices display the new target temperature (and also turn hvac_mode from off to heat if necessary).
however, if the target temperature changes at the underlying thermostat, it’s not changing in the versatile thermostat. only state changes (off/heat) are carried over.
Describe the solution you'd like
it would be nice if target temperature changes at the underlying thermostat could also be considerd by versatile thermostat
Describe alternatives you've considered
i could work around this with an automation, but the whole point of VT — as i see it — is to use less automations.
Additional context
i’m not sure if i made some kind of configuration error, but i tested it with serveral of my thermostats, but i can’t get VT to take the underlying thermostat’s target temperature.

turns out, it’s rather easy to automate:
alias: Heizung target temp. tado HK → VT
description: ""
trigger:
  - platform: state
    entity_id:
      - climate.tado_smart_radiator_thermostat_va41327xxxxx
    attribute: temperature
    for:
      hours: 0
      minutes: 0
      seconds: 10
    id: bad
  - platform: state
    entity_id:
      - climate.tado_smart_radiator_thermostat_va02783xxxxx
    attribute: temperature
    for:
      hours: 0
      minutes: 0
      seconds: 10
    id: kuche
  - platform: state
    entity_id:
      - climate.tado_smart_thermostat_ru24849xxxxx
    attribute: temperature
    for:
      hours: 0
      minutes: 0
      seconds: 10
    id: kind
  - platform: state
    entity_id:
      - climate.tado_smart_radiator_thermostat_va31056xxxxx
    attribute: temperature
    for:
      hours: 0
      minutes: 0
      seconds: 10
    id: schlafzimmer
  - platform: state
    entity_id:
      - climate.tado_smart_radiator_thermostat_va33157xxxxx
    attribute: temperature
    for:
      hours: 0
      minutes: 0
      seconds: 10
    id: stube
condition:
  - condition: template
    value_template: "{{( state_attr(trigger.entity_id,\"temperature\") | int(-1) ) >0 }}"
action:
  - service: climate.set_temperature
    data:
      temperature: "{{state_attr(trigger.entity_id,\"temperature\")}}"
    target:
      entity_id: "{{ hvac }}"
mode: parallel
max: 10
variables:
  room: "{{trigger.id}}"
  hvac: >-
    {% if room == "kind" %}climate.kind_lokal {% elif room == "stube"
    %}climate.stube_local {% elif room == "schlafzimmer"
    %}climate.schlafzimmer_lokal {% elif room == "kuche" %}climate.kuche_lokal
    {% elif room == "bad" %}climate.bad_lokal {% endif %}

Hello, I think you are right and target temp of VTherm does not follow th target temp of the underlying. This is because it is designed to be the exact inverse: VTherm controls the underlying through the use of preset temperatures. But it should be possible to set the target temperature and set the preset to manual in that case.
You are right, it is easy to automate but the mains purpose of the VTherm is to limitate as much as possible the automation.
I will have a look it is simple to implement and if it doesn't broke anything, and I keep you in touch. I think it should be easy to do.


Generic-alike type checks also on method calls

When calling a method annotated like following:
/**
 * @param stdClass[] $foo
 */
public function doBar(array $foo) {}

We should intercept the call and verify the type validity of $foo and of all parameters

@malukenho this is about simply using the ApplyTypeChecks to every parameter of every public method call :D

Implemented in #20 / #23


As a visitor, I want to see the activities of the app

On the home page, there should be shown some statistics of how much the app is used. Such as the number of messages, the number of sociables
Then people are able to see how popular this application is. Maybe this will motivate people to use it themselves
What do you think?

This may be something for the distant future. The chance is higher that people will not use the app, when they see how little it is used