format flexible docking

I want to do make a docking calculation with a non-standard molecule for which I am generating the flexible and ridid pdbqt files using my own scripts. After taking a look to the documentation I am testing Vina using the following flexible.pdbqt file:
BEGIN_RES SBE3    1
ROOT
ATOM      6  C2  SBE3    1      44.590  47.130  29.260  1.00  0.00     0.232 C
ENDROOT
BRANCH 6 7
ATOM      7  O2  SBE3    1      45.450  47.350  30.390  1.00  0.00    -0.642 O
ATOM      8  H2  SBE3    1      45.190  46.740  31.150  1.00  0.00     0.410 H
ENDBRANCH 6 7
ROOT
ATOM      9  C3  SBE3    1      45.210  47.240  27.870  1.00  0.00     0.143 C
ENDROOT
BRANCH 9 10
ATOM     10  O3  SBE3    1      46.210  46.210  27.830  1.00  0.00    -0.454 O
ATOM     11 C1X  SBE3    1      47.490  46.740  27.450  1.00  0.00     0.311 C
ATOM     15 C2X  SBE3    1      48.430  46.210  28.520  1.00  0.00     0.000 C
ATOM     16 C3X  SBE3    1      49.880  46.660  28.300  1.00  0.00     0.000 C
ATOM     17 C4X  SBE3    1      50.430  46.350  26.910  1.00  0.00     0.000 C
ATOM     18  SX  SBE3    1      51.020  44.660  26.590  1.00  0.00     0.842 S
ATOM     19 O1X  SBE3    1      51.010  44.520  25.100  1.00  0.00    -0.614 O
ATOM     20 O2X  SBE3    1      52.380  44.490  27.200  1.00  0.00    -0.614 O
ATOM     21 O3X  SBE3    1      49.990  43.750  27.190  1.00  0.00    -0.614 O
ENDBRANCH 9 10
ROOT
ATOM     12  C6  SBE3    1      41.770  48.010  26.090  1.00  0.00     0.232 C
ENDROOT
BRANCH 12 13
ATOM     13  O6  SBE3    1      40.700  47.110  26.430  1.00  0.00    -0.642 O
ATOM     14  H6  SBE3    1      40.120  47.560  27.100  1.00  0.00     0.410 H
ENDBRANCH 12 13
END_RES SBE3    1

but when I try to execute the docking calculation I get the following error:
PDBQT parsing error: Unknown or inappropriate tag found in flex residue or ligand.
 > ROOT

I have no experience on docking, so I guess there is some obvious error but I cannot find it. I would appreciate any help.

I believe from the error here, your flexible.pdbqt file was written incorrectly. There are specific tags the ligand.pdbqt and receptor.pdbqt must have to successfully start the docking prediction. If your source file is in the .pdb format, i highly suggest using autodock tools 1.5.7 (from the mgl tools suite here) to properly convert these files. Documentation on how to do this is also available online.

Before doing my own script for this I tried using MGLTools but then I have
no control on which parts of my molecule are ridig and which are flexible.
I followed the documentation of Vina to get my pdbqt and it seems correct
to me. This cannot be so complex... I would really appreciate if someone
could tell me which is the error in the lines I pasted in my original post.
El mar., 21 feb. 2023 1:29 a. m., Ritzhi-San @.***>
escribió:

I believe from the error here, your flexible.pdbqt file was written
incorrectly. There are specific tags the ligand.pdbqt and receptor.pdbqt
must have to successfully start the docking prediction. If your source file
is in the .pdb format, i highly suggest using autodock tools 1.5.7 (from
the mgl tools suite here https://ccsb.scripps.edu/mgltools/downloads/)
to properly convert these files. Documentation on how to do this is also
available online.
—
Reply to this email directly, view it on GitHub
https://github.com/ccsb-scripps/AutoDock-Vina/issues/179#issuecomment-1437699700,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AX2AARUYUEWUTZFXIHM5TH3WYQEALANCNFSM6AAAAAAVCIYIAM
.
You are receiving this because you authored the thread.Message ID:
@.***>

I found the error in the format: it seems that it cannot be 2 ROOT groups within the same residue. By grouping the atoms of the two ROOTs in the same ROOT I do not get the error. The correct format would be like this:
BEGIN_RES SBE3    1
ROOT
ATOM      6  C2  SBE3    1      44.590  47.130  29.260  1.00  0.00     0.232 C
ATOM      9  C3  SBE3    1      45.210  47.240  27.870  1.00  0.00     0.143 C
ATOM     12  C6  SBE3    1      41.770  48.010  26.090  1.00  0.00     0.232 C
ENDROOT
BRANCH 6 7
ATOM      7  O2  SBE3    1      45.450  47.350  30.390  1.00  0.00    -0.642 O
ATOM      8  H2  SBE3    1      45.190  46.740  31.150  1.00  0.00     0.410 H
ENDBRANCH 6 7
BRANCH 9 10
ATOM     10  O3  SBE3    1      46.210  46.210  27.830  1.00  0.00    -0.454 O
ATOM     11 C1X  SBE3    1      47.490  46.740  27.450  1.00  0.00     0.311 C
ATOM     15 C2X  SBE3    1      48.430  46.210  28.520  1.00  0.00     0.000 C
ATOM     16 C3X  SBE3    1      49.880  46.660  28.300  1.00  0.00     0.000 C
ATOM     17 C4X  SBE3    1      50.430  46.350  26.910  1.00  0.00     0.000 C
ATOM     18  SX  SBE3    1      51.020  44.660  26.590  1.00  0.00     0.842 S
ATOM     19 O1X  SBE3    1      51.010  44.520  25.100  1.00  0.00    -0.614 O
ATOM     20 O2X  SBE3    1      52.380  44.490  27.200  1.00  0.00    -0.614 O
ATOM     21 O3X  SBE3    1      49.990  43.750  27.190  1.00  0.00    -0.614 O
ENDBRANCH 9 10
BRANCH 12 13
ATOM     13  O6  SBE3    1      40.700  47.110  26.430  1.00  0.00    -0.642 O
ATOM     14  H6  SBE3    1      40.120  47.560  27.100  1.00  0.00     0.410 H
ENDBRANCH 12 13
END_RES SBE3    1

Now I have to see if the results are reasonable but at least I do not get the error in the format.

Looks like the branch 9 10 block could use a few more rotatable bonds.
Not sure what your goal is, but you could take a look at meeko's docking covalent ligands as flexible sidechains, maybe it will be helpful.

Hi Diogo
thank you very much for the suggestion. I took a look to the meeko package
and it seems interesting but I think I cannot use it because my molecule is
a bit large. What I have is a oligosaccharide that I build using my own
code, typically it has between 300 and 600 atoms. I am trying to develop a
code to have a reasonable docking approach for my specific molecules. Rigid
docking provides reasonable affinity constants and structures, so I decided
to include some level of flexibility. What I pasted in my previous posts is
just one of the residues of my polymer, but I typically have 14-16. I guess
Vina cannot manage all the torsions I would like to consider and this is
why I decided to allow rotations just at the beginning of the relatively
small chains representing mutations or sidechains in my molecule. The pdbqt
arising from my code seems to work well (at least it does not provide
errors and Vina is executed) but the resulting complexes and energies are
quite unreasonable... worse than those obtaining from the rigid docking...
I am working on this... so I would appreciate any advice.
On Tue, Feb 21, 2023 at 10:46 PM Diogo @.***> wrote:

Looks like the branch 9 10 block could use a few more rotatable bonds.
Not sure what your goal is, but you could take a look at meeko's docking
covalent ligands as flexible sidechains
https://github.com/forlilab/Meeko#docking-covalent-ligands-as-flexible-sidechains,
maybe it will be helpful.
—
Reply to this email directly, view it on GitHub
https://github.com/ccsb-scripps/AutoDock-Vina/issues/179#issuecomment-1439132786,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AX2AARX2QY5XLKF6RUXOBMTWYUZUNANCNFSM6AAAAAAVCIYIAM
.
You are receiving this because you authored the thread.Message ID:
@.***>

There are three categories of atoms in vina:

--receptor will not move at all
--ligand will translate and rotate as a rigid body, plus rotation of rotatable bonds
--flex rotation of rotational bonds, the atoms within the ROOT/ENDROOT block are fixed. This was designed for protein sidechains.
There may be multiple ligands (in separate files), multiple flexible residues (in a single file), and one receptor.

Rotatable bonds rotate all atoms downstream of the ROOT atoms. Each rotatable bond is associated with a BRANCH/ENDBRANCH block. See the example here on page 29 of the AD4.2 manual. Note how BRANCH 6 7 is nested inside the BRANCH 1 2 block.
Hope this helps.

Good to hear! I'll close, feel free to re-open if needed.

Yes, thank you for the help. Everything is working fine now with my code. I will explore now how to optimize the calculations but the results are reasonable now.
…
On Thu, Feb 23, 2023 at 6:03 PM Diogo @.> wrote: There are three categories of atoms in vina: - --receptor will not move at all - --ligand will translate and rotate as a rigid body, plus rotation of rotatable bonds - --flex rotation of rotational bonds, the atoms within the ROOT/ENDROOT block are fixed. This was designed for protein sidechains. There may be multiple ligands (in separate files), multiple flexible residues (in a single file), and one receptor. Rotatable bonds rotate all atoms downstream of the ROOT atoms. Each rotatable bond is associated with a BRANCH/ENDBRANCH block. See the example here on page 29 of the AD4.2 manual https://autodock.scripps.edu/wp-content/uploads/sites/56/2022/04/AutoDock4.2.6_UserGuide.pdf. Note how BRANCH 6 7 is nested inside the BRANCH 1 2 block. Hope this helps. — Reply to this email directly, view it on GitHub <#179 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AX2AARQ5RNTV4377CLTP663WY6J5FANCNFSM6AAAAAAVCIYIAM . You are receiving this because you authored the thread.Message ID: @.>

Dear Prof. Ángel Piñeiro,
did I guess properly that the molecule you are dealing with is sulfobutyl ether beta cyclodextrin?
If so, I would be very helpful for your advice. My goal is to dock small drug molecules into the cavity of this cyclodextrin. SBE side chains are very flexible so I am convinced that they should be treated as flexible parts of the receptor. I build the structure of SBE-b-CD manually in graphical moleular editor and thus obtained the pdb file without any headings such as ROOT or BRANCH. However, I cannot manage any flexible docking with this file, since I get the error: "IndexError: list index out of range". I cannot find the reason for this error and the issue I opened with a similar question ( #183) still remains unanswered. I wondered if it is only possible for macromolecules to ensure the flexibility of sidechains. However, your description shows that this is not the case. I would be very grateful if you could give me an advice how too deal with the flexibility of sulfobutyl ether sidechains. I would also ask if the results you obtained seem to be reasonable and the method of flexible docking can be succesful for the prediction of the affinities of a drug molecule to a cyclodextrin?


Fix exception that occurs when irtdb file does not exist

Usually the only way to fix an exception like this is to catch it when it happens.
If you are getting an exception when the file does not exist, then there will always be some other sort of exception that can happen if the file happens to be corrupted, or if you don't have permissions to read the file.
I am not familiar with the exception that you are trying to fix here? Would it be possible to add a try/catch around the spot where the exception is happening?

Usually the only way to fix an exception like this is to catch it when it happens.
If you are getting an exception when the file does not exist, then there will always be some other sort of exception that can happen if the file happens to be corrupted, or if you don't have permissions to read the file.
I am not familiar with the exception that you are trying to fix here? Would it be possible to add a try/catch around the spot where the exception is happening?

You're right, it's better to catch other exeptions like the database being corrupted. This is an exception that happens when closing peptide settings, it tries to check if it needs to add iRT standards to the document. I think it's better in this case to just ignore rather than show an error.


Install prompt inform user...

It poor connectivity conditions it takes a while until the app is installed, and this will confuse users who can't find the app after click install. Can we inform the user when the app is installed?  If not possible, just add a message in the prompt that it will take a few seconds

don't think this would be possible on iPhone, do android apps notify you when they've been installed? On iPhone we're shown the screen where we can see it downloading, what's the flow on android? We could do something similar maybe?

I guess it depends on the device settings and android, play store versions etc.


[FEATURE]: add support for multiple profiles during buildtest build

Please describe your feature
We should introduce the concept of profiles which will mimic the options that can be encapsulated into a single name to avoid having users to type several options. This will address this issue where we have multiple runs
buildtest bd -b /tmp/foo.yml -b tutorials
buildtest bd -b  /tmp/containers/ --rebuild=2 -x /tmp/containers/bar.yml
buildtest bd -t python
buildtest bd -t containers -m gcc/9.3.0 --module-purge --report /tmp/containers.json

Instead of having to run the above commands we can add profiles that can be defined in configuration file as follows
profiles:
  python:
    tags: [python]
  containers:
    tags: [containers]
    module: [gcc/9.3.0]
    module-purge: True
  foo:
    buildspecs: [/tmp/foo.yml , tutorials ]
  bar:
    buildspecs: [ /tmp/containers/ ]
    rebuild: 2
    exclude: [ /tmp/containers/bar.yml ]
    report: /tmp/containers.json

With this being said we can simply run the commands from the profiles where each profile sub-key must be a unique name and must be alphabets only so the following statements are equivalent
The two commands below will behave the same way.
buildtest bd -b /tmp/foo.yml -b tutorials
buildtest bd --prof=foo

If other options are specified when --prof is used they will be simply ignored.
Suggest potential solution
No response
Additional Information
No response
Post question in Slack

[X] I agree that I posted my question in slack before creating this issue

Is there an existing issue

[X] I confirm there is no existing issue for this issue

@prathmesh4321 @Xiangs18 what do you guys think about this feature?

@prathmesh4321 @Xiangs18 what do you guys think about this feature?

Hi, @shahzebsiddiqui this feature looks really helpful and I think we should implement this. I have one concern though. The user would have to remember the names of the profiles and might often forget the names and in that case would have to scan through the complete list of profiles.
So I think we can also have something like buildtest cg --prof=python --json which will display an individual profile from the available list of profiles.
Additionally, we can have a command to filter the profiles that contain a particular option. So for example in the below configuration:
profiles:
  python:
    tags: [python]
  containers:
    tags: [containers]
    module: [gcc/9.3.0]
    module-purge: True
  foo:
    buildspecs: [/tmp/foo.yml , tutorials ]
    exclude: [ /tmp/containers/xyz.yml ]
  bar:
    buildspecs: [ /tmp/containers/ ]
    rebuild: 2
    exclude: [ /tmp/containers/bar.yml ]
    report: /tmp/containers.json

if we do something like buildtest cg profiles --filter exclude it should list the following profiles that contain the exclude option. So it becomes easier to choose from the filtered profiles even if the user doesn't remember the individual profile names.
foo:
   buildspecs: [/tmp/foo.yml , tutorials ]
   exclude: [ /tmp/containers/xyz.yml ]
 bar:
   buildspecs: [ /tmp/containers/ ]
   rebuild: 2
   exclude: [ /tmp/containers/bar.yml ]
   report: /tmp/containers.json

i did post an issue https://github.com/buildtesters/buildtest/issues/1490 pertaining to listing profiles. As per --filter i am not entirely sure if that will be useful and it may get a bit complicated to setup. I think that simply seeing the profile via command line is very helpful so if one doesn't know the attributes they can get them as pose to running buildtest cg profiles --filter exclude and not get what they want since chances are if output returns nothing its not useful.

One command that i think will be useful is the ability to remove profiles since --save-profile will add them, the only way to remove them right now is manually do them so i do propose a builtin command such as buildtest cg profile remove <profile> to remove profiles and update configuration file


Adding todo

Just leaving a todo comment re: DatePicker UX

Hi there. What is the purpose of this PR?

Hi,
As part of the application process for Revelry. Also, when looking at Revelry’s DatePicker I noticed that it was a) too fast with the open/close animation; b) not smooth when opening/closing. Thought I’d use a @todo comment to message the devs.

Richard

On Feb 8, 2018, at 10:23 AM, Robert Prehn<EMAIL_ADDRESS>wrote:
Hi there. What is the purpose of this PR?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/revelrylabs/possum/pull/137#issuecomment-364165357, or mute the thread https://github.com/notifications/unsubscribe-auth/AVXZY7_K7p9AxH5pzcvgavNdpk9Ck40Yks5tSx95gaJpZM4R9aab.

@richardschwalb If there aren't code changes for this yet, could you log an issue here https://github.com/revelrylabs/possum/issues?

Ok, just did. Thx.

On Feb 8, 2018, at 10:50 AM, Robert Prehn<EMAIL_ADDRESS>wrote:
@richardschwalb https://github.com/richardschwalb If there aren't code changes for this yet, could you log an issue here https://github.com/revelrylabs/possum/issues https://github.com/revelrylabs/possum/issues?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/revelrylabs/possum/pull/137#issuecomment-364174559, or mute the thread https://github.com/notifications/unsubscribe-auth/AVXZY80kqVGEx8wI77-MhJbR-fX2duIpks5tSyXIgaJpZM4R9aab.

Closing since #138 captures the todo now.


Reflection for missing filenames

@dcramer @Jean85 how does this look? It changes the stacktrace for Laravel (5.1 tested) from:

To:

Which is more correct (considering the app path is set on both correctly).

Seems good to me, but I'm not sure about the impact in non-Laraver apps...

Assuming the class is available in the frame this should be the same improvement.
If we know the class we can extract the file and then enhance the in-app detection.
P.S. I replaced the first screenshot with the current one.


Error importing paddleclas after install

i got error
local variable 'total_width' referenced before assignment

when use import paddleclas for the first time after installation
i use branch develop and using guide from readme_en.md
error gone when i change the total_width to arbitary int number
https://github.com/PaddlePaddle/PaddleClas/blob/95edc0d6d3f3e1cc299a7f58012177769fc63fb4/paddleclas.py#L352
after i change,,i got another error that say
AssertionError: logger should not be initialized twice or more.

error gone when i comment this line https://github.com/PaddlePaddle/PaddleClas/blob/95edc0d6d3f3e1cc299a7f58012177769fc63fb4/ppcls/utils/logger.py#L42
and after that, the paddleclas work perfectly
maybe you guys can take a look before merging #2036

Thanks for reporting bug. It will be solved by PR #2084


Add snippet on how to create a did:dht identifier

Type
Docs Bug Report
Feedback
It would be nice to have a little sample how to create a did:dht like this:
`//did:dht
import { DidDhtMethod } from '@web5/dids'
//Creates a DID using the DHT method and publishes the DID Document to the DHT
const didDht = await DidDhtMethod.create({ publish: true });
//DID and its associated data which can be exported and used in different contexts/apps
const portableDID = JSON.stringify(didDht);
//DID string
const did = didDht.did;
//DID Document
const didDocument = JSON.stringify(didDht.document);
//Cryptographic keys associated with DID
const keys = JSON.stringify(didDht.keySet);
//Primary form of a DID. more info: https://www.w3.org/TR/did-core/#dfn-canonicalid
const canonicalId = didDht.canonicalId;
console.log(didDht);`
Screenshots
Paste relevant images or screenshots here
developer.tbd.website URL
No response
Browser
No response
Relevant log output
No response

will need a package.json to try this like this:
{
  "name": "diddht-try",
  "dependencies": {
    "@web5/dids": "^0.2.4"
  },
  "type" : "module"
}
``` then `npm install` - and `node index.js`

Never mind, you already have it https://developer.tbd.website/docs/web5/build/decentralized-identifiers/how-to-create-did


[AutoRelease] t2-netapp-2022-07-21-97452(Do not merge)

https://github.com/Azure/sdk-release-request/issues/3018
Live test fail, detailed info is in pipeline log(search keyword FAILED)!!!
https://dev.azure.com/azure-sdk/internal/_build?definitionId=1015

issue link:https://github.com/Azure/sdk-release-request/issues/3018


Refactor refactor the code of runtime filter

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall


add feature: add -e,--external to f2 predict

Should be able to pass in externally derived GFF3 gene model predictions, ie from newer AI/DL/ML methods like helixer and/or Tiberius.  But the requirements are too onerous to include as a dependency.  Idea is that GFF3 can be passed via CLI, use gfftk sanitize to clean up the input and then add to predictions pool.

If we do a nextflow style workflow assume that could run these separately
and pass in. Better to keep the core lightweight as you have done!
Sent from Gmail Mobile
Jason Stajich - @.***
On Wed, Oct 30, 2024 at 2:36 PM Jon Palmer @.***> wrote:

Should be able to pass in externally derived GFF3 gene model predictions,
ie from newer AI/DL/ML methods like helixer and/or Tiberius. But the
requirements are too onerous to include as a dependency. Idea is that GFF3
can be passed via CLI, use gfftk sanitize to clean up the input and then
add to predictions pool.
—
Reply to this email directly, view it on GitHub
https://github.com/nextgenusfs/funannotate2/issues/3, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAL5OZTAGN6ZI6E3ZJNCE3Z6FGN7AVCNFSM6AAAAABQ5BOUZCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYZDKMZZHA2DINQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Yeah needs to be as dependency-light as we can make it. I'm leaning towards not building in external tools for parsers/etc for f2 annotate as well, it's too hard to keep updating when tools change. Instead, just allow for the 3 column TSV format to add any annotation of choosing. I've kept in some of the basics (pfam, cazy, merops, uniprot, busco) -- but not going to do emapper, interpro, signalp, antismash, etc.

completed


Rpt. agreements by cancellation dates

#506

It is the first part of the Rpt. The report is made for MetaDB. The rest of the POL and invoice lines follow as separate tables. The POL uuid was also specified in this report so that the rest can be added if necessary.

Accepted improvement suggestion and left additional comment on the filters.
Many Thanks @lib-eric

Additional requests for the filters have been added.
Thanks @adoerrer

I see the description of this query as "report of agreements by cancellation dates."  Is there any other description or context that could be added?

I see the comment:

It is the first part of the Rpt. The report is made for MetaDB. The rest of the POL and invoice lines follow as separate tables. The POL uuid was also specified in this report so that the rest can be added if necessary.

What is the overall report that this is the first part of?

It's a bit confusing. "Report of agreements by cancellation dates" is the correct description for this report. Originally, the requirements were to add the data from the order and invoice too. After discussing this, we decided to split the report into 3 parts. The part with the agreements can be found here. The other parts (order and invoice data) are created as separate derived tables. The derived tables for orders and invoices will be implemented in https://github.com/folio-org/folio-analytics/issues/518 ; https://github.com/folio-org/folio-analytics/issues/517 ; https://github.com/folio-org/folio-analytics/issues/516 . Therefore, the UUID for the POL is also included in this report.

Thanks.  I'm looking for a complete description of the changes to put into the merge commit.  I think I can quote your last comment.

Can we include any documentation for the readme file?

I will update the readme file at sql_metadb/report_queries/erm/resource_lists/.


[Proposal] Fix wrapper table

Currently, the individual wrappers are documented in a gigantic table here: Wrappers - Gym Documentation. The table is confusing. Should we have one auto-generated page per wrapper (not including the abstract super-classes) and list them on the main page?

@Markus28

In the near future, we will be releasing a new set of wrappers to replace the old ones. So it would be nice to have some auto gen code to use the docstrings for the wrappers. So it might to worth using that code for the current wrappers as well
These wrappers will be split into several helpful categories, lambda observation wrappers, step wrappers, stateful wrappers, etc. My plan is for these wrappers to have large and well-documented docstrings so I think a page for each of these wrapper categories will be necessary.


Fix #266: check if i is in states; avoid indexing

Hi @EelVex! I lack a bit of context here. Why do you request this change?
Using if i not in states has more time complexity than if states[i] != i (the first option needs to go through all the list in the worst case, while the second option has to check only one value). Why do we want to avoid indexing?

Hi @AlbertMitjans. This fixes the bug #266 where an IndexError is triggered when the max i included in states is less than  number_of_states-1. For example if states = [1, 2, 5] and number_of_states = 8 the check states[6] != 6 can not be performed.
This PR tries to fix the bug using the fewest modifications possible from the original. If time complexity is of an issue then another fix could be:
if len(states) < number_of_states:
    missing = set(range(number_of_states)) - set(states)
    for i in missing:
        probs.insert(i, 0.0)

or, perhaps even better, the loop could be replaced with a np.insert(np.array(probs), missing, 0.0)

the loop could be replaced with something like np.insert(np.array(probs), missing, 0.0)

@EelVex this would be much better! Can you implement it?

Also please add a new test testing that the bug is fixed

the loop could be replaced with something like np.insert(np.array(probs), missing, 0.0)

@EelVex this would be much better! Can you implement it?

@AlbertMitjans sure. New PR or in this one?

New PR or in this one?

@EelVex in this one!

@AlbertMitjans the last commit makes the function more readable and much more efficient. I'm not sure where (and if) testing is needed.

Hi @EelVex ! This looks much better, thanks!!
Please add a test in tests/test_runtime.py in the class TestSampler, testing that the bug that you reported is fixed.
Also add an entry in CHANGELOG.md under Bug fixes explaining in one line the bug that you just fixed.

This is still giving problems (with version 0.32.0) when not all the states are measured. Is this PR going to be merged in the future?

Fixed in #337


question about the pre-trained weights

Dear authors,
Thanks for the great work! I have downloaded the pre-trained weights of ViT-B-16(224)+RoBERTa checkpoint from https://github.com/zdou0830/METER/releases/download/checkpoint2/meter_clip16_224_roberta_pretrain.ckpt, and found that the last layer of the visual encoder "vit_model.visual.transformer.resblocks.11..." is not included in the ckpt file? Did I miss something? Could you please help me to check it?

Hi, we removed the last layer of CLIP (https://github.com/zdou0830/METER/issues/9) because it is essentially a pooling layer and we found that removing it could lead to small improvements.

Hi, we removed the last layer of CLIP (#9) because it is essentially a pooling layer and we found that removing it could lead to small improvements.

many thanks for your reply~


typescript infer type of 'this' wrongly

Bug Report
🔎 Search Terms
decorator, type inference
🕗 Version & Regression Information
all of the versions
⏯ Playground Link
Playground
💻 Code

export interface SimpleChange<T> {
  firstChange: boolean;
  previousValue: T;
  currentValue: T;
  isFirstChange: () => boolean;
}

export function OnChange<T = any>(
  callback: (simpleChange?: SimpleChange<T>) => void
) {
  const cachedValueKey = Symbol();
  const isFirstChangeKey = Symbol();
  return (target: any, key: PropertyKey) => {
    Object.defineProperty(target, key, {
      set: function (value) {
        console.log(this); // <----- here type of `this` is 'A'

        if (this[isFirstChangeKey] === undefined) {
          this[isFirstChangeKey] = true;
        } else {
          this[isFirstChangeKey] = false;
        }
        // No operation if new value is same as old value
        if (!this[isFirstChangeKey] && this[cachedValueKey] === value) {
          return;
        }
        const oldValue = this[cachedValueKey];
        this[cachedValueKey] = value;
        const simpleChange: SimpleChange<T> = {
          firstChange: this[isFirstChangeKey],
          previousValue: oldValue,
          currentValue: this[cachedValueKey],
          isFirstChange: () => this[isFirstChangeKey],
        };
        callback.call(this, simpleChange);
      },
      get: function () {
        return this[cachedValueKey];
      },
    });
  };
}

class A {
  // convert the arrow function to the normal function to make it works
  @OnChange<number>((simpleChange)=> {
    this.print('n', this.n); // <-----  this is unknown here
  })
  n = 0;

  @OnChange<number>(function (simpleChange) {
    this.print('z', this.z); // <----- this is of type any
  })
  z = 0;

  print(source: string, value: number) {
    console.log(source, value);
  }
}

const a1 = new A();
a1.n = 2;
a1.n = 2;
a1.z = 3;
a1.n = 3;
a1.z = 3;
a1.z = 3;
console.log(a1);

🙁 Actual behavior
the type of this should be inferred correctly
this is respect to this https://stackoverflow.com/questions/70595096/tyepscript-set-type-of-this-on-function-decorator
based on the position that the this is called on the callback, the this should be inferred.
it's similar to what mentioned in https://github.com/microsoft/TypeScript/issues/37300#issuecomment-787493352
but that thread is closed and have slightly different reasoning, but the issues are related and I expect those also be addressed
also note that the behavior is against the documentation, at least not how the difference between the arrow function and normal function is described.
moreover, the type of target is known by the typescript. if I call the callback like this the typescript should infer the type correctly
callBack.call(target, args);

🙂 Expected behavior
the type of this inferred as part of the decorator signature
arrow functions shouldn't throw an error

Please provide a minimal example and identify exactly what you believe to be the bug here. There's a lot of extraneous code here and multiple uses of this and it's not clear what you're trying to get at.

@RyanCavanaugh I provided the playground, what else do you need? first, in the playground you can see that multiple errors appears and there is nothing wrong with the code the TS is showing error. that's one of the bug.
second the type of this in both decorators are wrong, one is any another is unknown that is wrong too.


Added SMS-Classification

Have you read the Contributing Guidelines ?
Yes.
Description
I have read all the guidelines and followed them.
Fixes #issue_no
#983

Checklist

[✔ ] I've read the contribution guidelines.
[ ✔] I've checked the issue list before deciding what to submit.
[✔ ] I've edited the README.md and link to my code.

Related Issues or Pull Requests
N/A

Ok i will do it by evening. Sorry

Have completed

Requesting you to check it once

Sorry i created another pull request, I have completed my work

Added SMS Classification in Basic


[BUG] model.predict() method for MLP/DLRM/DCN models throw exception

Bug description

model.predict() method for MLP/DLRM/DCN models throws exception when trying to predict probabilities on the test file.
Steps/Code to reproduce bug

Download the sample dataset
Run the following code below. I am getting valueerror whose screenshots have been attached here.
Code to Reproduce   os.environ["TF_GPU_ALLOCATOR"]="cuda_malloc_async"
   import cudf
   import glob
   import gc
   import pandas as pd
   import tensorflow as tf
   import warnings
   warnings.filterwarnings("ignore")

   import nvtabular as nvt
   from nvtabular.ops import *
   from fit_transform import workflow_fit_transform

   from merlin.schema.tags import Tags
   from merlin.schema import Schema

   import merlin.models.tf as mm
   import merlin.models.tf.dataset as tf_dataloader

   from merlin.io.dataset import Dataset
   from merlin.schema.io.tensorflow_metadata import TensorflowMetadata
   from merlin.models.tf.blocks.core.aggregation import CosineSimilarity

   train_path = '/workspace/data/train/*.parquet'
   valid_path = '/workspace/data/valid/*.parquet'
   test_path = '/workspace/data/test/*.parquet'
   output_path = '/workspace/data/processed'
   model_path = '/workspace/data/processed/models/'

   user_id = ["userid"] >> AddMetadata(tags=[Tags.USER_ID]) >> Categorify(freq_threshold=5)
   item_id = ["contentid"] >> AddMetadata(tags=[Tags.ITEM_ID]) >> Categorify(freq_threshold=5)
   targets = ["relevant"] >> AddMetadata(
   tags=[str(Tags.BINARY_CLASSIFICATION), "target"])

   add_feat = ["eventtype", "eventStrength", "BusinessArea", "JobPosition","HrRoleName", "Location"] >> nvt.ops.Categorify()

   te_feat = (
       ["userid", "contentid"] + add_feat >>
       TargetEncoding(
           ['relevant'],
           kfold=1,
           p_smooth=20
       ) >>
       Normalize()
   )

   outputs = user_id+item_id+targets+add_feat+te_feat

   workflow = nvt.Workflow(outputs)

   train_dataset = nvt.Dataset(train_path)
   valid_dataset = nvt.Dataset(valid_path)
   test_dataset = nvt.Dataset(test_path)

   workflow.fit(train_dataset)
   workflow.transform(train_dataset).to_parquet(output_path=output_path + "/train/")
   workflow.transform(valid_dataset).to_parquet(output_path=output_path + "/valid/")
   workflow.transform(test_dataset).to_parquet(output_path=output_path + "/test/",shuffle=False)

   workflow.save('workflow')

   schema = TensorflowMetadata.from_proto_text_file(output_path + '/train/').to_merlin_schema()
   target_column = schema.select_by_tag(Tags.TARGET).column_names[0]

   model2 = mm.MLPBlock([64, 32]).to_model(
       schema,
       prediction_tasks=mm.BinaryClassificationTask(target_column)
   )

   model2_description = "MLP"

   model3 = mm.DLRMModel(
       schema,
       embedding_dim=64,
       bottom_block=mm.MLPBlock([128, 64]),
       top_block=mm.MLPBlock([128, 64, 32]),
       prediction_tasks=mm.BinaryClassificationTask(target_column)
   )

   model3_description = "DLRM"

   model4 = mm.DCNModel(
       schema,
       depth=2,
       deep_block=mm.MLPBlock([64, 32]),
       prediction_tasks=mm.BinaryClassificationTask(target_column)
   )
   model4_description = "DCN"

   model2.compile('SGD', run_eagerly=False)
   model3.compile('SGD', run_eagerly=False)
   model4.compile('SGD', run_eagerly=False)

   batch_size = 16*1024
   train_dl = tf_dataloader.BatchedDataset(
       Dataset(output_path + '/train/*.parquet', part_size="500MB"),
       batch_size = batch_size,
       label_names = [target_column],
       shuffle= True,
       schema = schema,
   )

   valid_dl = tf_dataloader.BatchedDataset(
       Dataset(output_path + '/valid/*.parquet', part_size="500MB"),
       batch_size = batch_size,
       label_names = [target_column],
       shuffle = False,
       schema = schema,
   )
   model2.fit(train_dl, validation_data=valid_dl)
   model3.fit(train_dl, validation_data=valid_dl)
   model4.fit(train_dl, validation_data=valid_dl)

   model2name = model_path + model2_description
   model3name = model_path + model3_description
   model4name = model_path + model4_description

   model2.save(model2name)
   model3.save(model3name)
   model4.save(model4name)

   test = pd.read_parquet("/workspace/data/processed/test/part_0.parquet")
   test_input = test[['contentid', 'userid', 'eventtype', 'eventStrength','BusinessArea','JobPosition', 'HrRoleName', 'Location']]

   def tf_tensor_dict(df):
       data = df.to_dict("list")
       return {key: tf.convert_to_tensor(value) for key, value in data.items()}

   test_tensor = tf_tensor_dict(test_input)

   test['predict_proba_MLP'] =  @model2.predict(test_tensor)

### Expected behavior
<!-- A clear and concise description of what you expected to happen. -->

### Environment details
- Merlin version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):

### Additional context
<!-- Add any other context about the problem here. -->
![Screen Shot 2022-04-04 at 9 17 34 AM](https://user-images.githubusercontent.com/28987826/161588939-8a61431b-cba3-47e6-926a-0ddbf676caca.png)
![Screen Shot 2022-04-04 at 9 18 00 AM](https://user-images.githubusercontent.com/28987826/161588954-0cf798b4-939e-405e-a31d-aeeca8883592.png)

@mamtaed8811 May be  you can  update your main branch  to explore the latest examples:
cd /models
git checkout main
git pull origin main
pip install -e .

then you can build your models with the updated nbs (see 03-Exploring-different-models) .
For the error you are getting, pls get the batch from you dataloader and check if the dtypes of your model input are matching with the dtypes  of the input features in the test_tensor, if not make the dtypes of test_tensor as same in your batch.
import merlin.models.tf.dataset as tf_dataloader
train_dl= tf_dataloader.BatchedDataset(
        Dataset(output_path + '/train/*.parquet'),   # if you are using new examples this line should be only  `train`.
        batch_size=32,
        shuffle=False,
    )
batch = next(iter(train_dl))
batch

@rnyak I don't quite follow your comment inside the code block.
My follow up questions:

train_dl= tf_dataloader.BatchedDataset(
Dataset(output_path + '/train/*.parquet'),   # if you are using new examples this line should be only  train.
batch_size=32,
shuffle=False,
)
batch = next(iter(train_dl))
batch```
In the above code, for my test dataset, am I to reuse it like this below?
test_dl= tf_dataloader.BatchedDataset(
        Dataset(output_path + '/test/*.parquet'),   # if you are using new examples this line should be only  `train`.
        batch_size=32,
        shuffle=False,
    )
batch = next(iter(test_dl))
batch

"# if you are using new examples this line should be only  train"
What am I supposed to be passing inside Dataset for new dataset i.e test files?

@rnyak I am creating a dataloader for test files like in my screenshot and then reading the actual test parquet file where I am filtering out the unwanted columns and then comparing the dtypes. Obviously, the dtypes won't match because NVTabular is converting the categorical columns into integer type for the batch. My question here is how do I use a stored test parquet file(unseen data for the model) for inference. What's the right way of doing it? This definitely is not the right way as it's not working currently. Could you please reproduce it on your end and guide me on the right way to use the API for inferencing?

@mamtaed8811 you need to use the preprocessed test set to feed to model. you cannot feed raw test set to model.  use the one processed from NVTabular.

I am not seeing this issue anymore with the newest container. Thank you! I will close it now.


Backup method bookstack

Attempted Debugging

[X] I have read the debugging page

Searched GitHub Issues

[X] I have searched GitHub for the issue.

Describe the Scenario
Hello sir,
I have my own method to back up my data in bookstack app. The method is that I'm navigate to the volume of my bookstack container and after that I zip it. So if I run into a problem, I just need to create a new bookstack container and unzip my previous bookstack volume, and map it with portainer.
Is my method correct sir ?.
Exact BookStack Version
v22.07
Log Content
No response
PHP Version
No response
Hosting Environment
Oracle Cloud

Is my method correct sir ?.

There's no way to ensure a backup process works until you attempt to restore.
Try to create a separate running instance of your BookStack system, from your backup data, to be sure.
You only mention zipping of the bookstack container volume, you'll also need the data from the database container.
Zipping live database data can be a little risky since you're working with active data. Ideally you'd create a mysqldump of the data. We have details on creating a database dump here but the approach may differ if attempting via a container.

Hi sir, I tried and it worked.
Thank you


sessionID's are not unique

From @weAllWeGot on December 15, 2016 15:4
there is some weird overlap between sessions. and they influence the users conversations
Copied from original issue: weAllWeGot/kbai_chat_bot3#29

http://flask.pocoo.org/snippets/category/sessions/

https://www.tutorialspoint.com/flask/flask_sessions.htm


fixed quote generator for mobile version

"authorname" should be set in order to work on mobile version, also "author" is not needed at all now.
Tested on desktop version <IP_ADDRESS> and mobile <IP_ADDRESS>5

Thanks for this pull request. I've merged it and will push the new version out soon.


No In-memory implementation for ContinuationTokenSpecification

Using ContinuationTokenSpecification on InMemoryRepository throws NotImplemented exception.
We have been using InMemoryRepositories for unit testing the repository calls. It recently came to our notice when we were using ContinuationTokenSpecification that it does not have in-memory implementation yet.
if (specification.UseContinuationToken)
{
    throw new NotImplementedException();
}

Are we looking to add this feature any soon and if there are any alternatives please? Thanks!

Hi @khingda-next I would advise setting up a test instance of Cosmos DB either in the cloud or running as the emulator, the intricacies of the implementation of continuation tokens in our opinion could not be replicated well enough in the in memory store hence the reason we chose to just throw a not implemented exception here.
You can either require the emulator running on your machine, run it in docker or point your tests at a cloud instance see these docs:
https://learn.microsoft.com/en-us/azure/cosmos-db/local-emulator?tabs=ssl-netstd21

Thanks for your response!


SGE scripts

Hi,
Does anybody uses SGE queing system ??? Actually the code runs on LSF. Would be great to have SGE calls also ?

I have adapted the create_jobs.py and setupDirs.py scripts for SGE on our cluster.  This won't work automatically for you, but it should be much closer to being in working order on SGE than the original LSF scripts.  Just replace the original copies of these scripts in the LSFScripts/ directory with these versions.  I've replaced all the options and it works on my cluster, but you'll have to at least change the queue name ('micro') with your own queue name.  Not sure if there are other changes you need to make...
https://github.com/russianconcussion/lsa.sge.scripts.git

Thanks !!! that´s great.
To quickly replace the (´micro´) queuing name i just write a sed line as follows
sed "s/micro/myqueue/g"  create_jobs.py > out1
sed "s/micro/myqueue/g"  setupDirs.py > out2
thanks !!!

Glad to help.  Let me know if it doesn't work.

By the way, how do you handle the HashCounting.sh parallel_gnu with SGE ???
The only way i food was to write a qsub script and call the HashCounting.sh script from inside. I don´t know how much memory need to be allocated, depending on the hashsize:
scriptSGE.sh
#$ -cwd
#$ -l  mem=28G
#$ -o out
#$ -e err
#$ -q queuelocal
hashSize=22
python LSA/create_hash.py -i original_reads/ -o hashed_reads/ -k 33 -s $hashSize > Logs/CreateHash.log
did you find a better way ??

I'm not sure that I even ran that script.  I followed the large dataset analysis instructions at http://latentstrainanalysis.readthedocs.org/en/latest/large_collections.html.  The instructions never said to run that script, and I don't have any record of running that script in my logs.  Sorry, but I can't help there!


Added html messages for all 3 scenarios 2h

We will need to insert the link in the message once the endpoints are ready

#65


Making the library esp idf compatible (no Arduino libraries)

Hey there,
that's rather a feature request than an issue. First of all, thanks for the cool library. I think it would be super awesome to make the code Arduino independent (esp idf only). Have you ever thought about porting it?
Best, Timm

That's a great idea. It should not be difficult. The challenge here is that I'm not used to program using IDF directly so I do not have the skills to do the integration. Feel free to do a fork or a pull request


LoggerMiddleware not outputting anything

When I run titiler with debug enabled:
$ TITILER_API_DEBUG=1 uvicorn titiler.main:app --reload

I expect to see something printed in stdout but there is nothing.

can you try with --log-level debug ?

same thing:
$ TITILER_API_DEBUG=1 uvicorn titiler.main:app --reload --log-level debug
INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [24825] using watchgod
INFO:     Started server process [24827]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     <IP_ADDRESS>:44610 - "GET /cog/info?url=https%3A%2F%2Fgithub.com%2Fmapbox%2Frasterio%2Fblob%2Fmaster%2Ftests%2Fdata%2FRGB2.byte.tif%3Fraw%3Dtrue HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:44984 - "GET /healthz HTTP/1.1" 200 OK

@vincentsarago what do you think about this fix?
diff --git a/titiler/middleware.py b/titiler/middleware.py
index d3c5841..9b90c41 100644
--- a/titiler/middleware.py
+++ b/titiler/middleware.py
@@ -58,6 +58,7 @@ class LoggerMiddleware(BaseHTTPMiddleware):
         super().__init__(app)
         self.logger = logger
         logger.setLevel(logging.DEBUG)
+        logger.addHandler(logging.StreamHandler())

         self.querystrings = querystrings
         self.headers = headers

@drnextgis I think this is a Uvicorn bug.
ref: https://github.com/tiangolo/fastapi/issues/2019
The current implementation works fine when not using uvicorn (e.g. in AWS Lambda)
I've been able to print the logs by using a config file
version: 1

disable_existing_loggers: False

formatters:
  console:
    format: "%(levelname)s\t%(name)s\t%(message)s"

handlers:
  console:
    class: logging.StreamHandler
    formatter: console
    stream: ext://sys.stderr
    level: 0

root:
  handlers: [ console ]
  level: 0

TITILER_API_DEBUG=TRUE uvicorn titiler.main:app --reload --log-config log-config.yaml
ref: https://github.com/encode/uvicorn/issues/944
I'm going to close this issue because I believe this is not related to TiTiler itself.


Don't go through Display for &str tracing fields

The extra % was piping the values through tracing::field::display, which makes little sense for &strs, and would lead to quotes being omitted in places where they would be helpful (for example tracing_subscribers fmt::Subscriber).

Not specifying the value will just reserve empty fields for them, so it should be path = path in the macro to actually use the values. It's a footgun with tracing's current defaults.

I thought I used this style successfully before. I'll test it later.

Dang, you're right. I will send a PR to fix it soon.


[feature]: Opt-in Opt-out chrome synch

Done in https://github.com/dvcol/synology-download/releases/tag/v2.2.0


Tags does not create decks

at the time of reviewing flashcard, the assigned tags are not being created as decks.
Is there something I am missing? because i think if i add tags then it would reflect as decks at the time of review. Or should i add tags in YAML

Before raising this issue i didn't added tags in the settings, but i did after u mentioned here. Still, I can only see two tags as decks, the rest are not being shown. I am extremely sorry I misunderstood the way these tags works, I thought it is intuitive means after adding #flashcards I can add different tags like neuracache do with their software.

One more thing i would like to show, which i think if it can be implemented then life would be very easy, it would be a delight to create flashcards.

see in this screenshot, i have tried to create a flashcard within this block means my outliner plugin was active, this does not reflect in my flashcard. If flashcard within sub child could have been created, then it would be awesome

You can change the settings to have the flashcards scheduling placed in the last line of the flashcard's text. That should solve this problem.

So, for the decks, the plugin picks only the first matching tag and uses that for the deck.

as you can see in this screenshot i have added different tags but only first two tags are shown. The rest of the tags are not shown as decks, so is it how it works? I am just asking, i am not aware about it.
And related to the settings to have the flashcards scheduling, it is already turned on. Still, it does not reflect in my workflow.

I had the same problem, make sure you haven't press enter accidentally. If you press enter the tags are not read correctly.

I've made a fix to allow splitting with arbitrary numbers of any whitespace character, newlines included. That should fix what @Couriuss raised.

Hey @semits20, could you give us an update on this?

I just installed the plugin and was having a very similar issue. I ended up figuring out a workaround. To repro:

Install the plugin with default settings.
Add #flashcards in an existing.
Add Cloze deletions (the disable cloze cards? option is not enable). I put three on a single line.
Run the Spaced Repetition: Review flashcards command
Observe that there are no decks in the dialogue.
Add a single line flashcard using the default separator ::
Repeat step 4
Observe that the Cloze deletions AND the single line flashcard have now appeared.

@Masstronaut, there's a bug with the cloze cards. It requires a blank line after the card text. I'll fix this.

Closing

I found that it was only preexisting tags that didn't show up. If I changed my #History to #HistoryCards (in both the settings and the note) and used that, it worked.


All supported languages must be selected for new version

deployed to Prod

Detect whether the SDK is first beta https://github.com/Azure/azure-sdk-tools/issues/5333#issuecomment-1440951857


Fixed a regression in 0.2 by adding back the default sequence generator

The default in-memory sequence generator is now added in the plugin descriptor doWithSpring. This is how it was implemented in version 0.1 of the plugin.
This makes the url shortener work out of the box. However the default generator should not be used in production deployments.

Thank you!


Undefined symbols for architecture x86_64

Hello,
I use latest xcode version, i created a new project swift, added the package with swift package manager and now the build is failing :
Undefined symbols for architecture x86_64:
"_inflate", referenced from:
SignalRClient.(Inflater in 0DEB592A83220160D0A867EC9A8F696A).inflate(: Swift.UnsafePointer<Swift.UInt8>, length: Swift.Int, final: Swift.Bool) throws -> (p: Swift.UnsafeMutablePointer<Swift.UInt8>, n: Swift.Int) in SignalRClient.o
"_zlibVersion", referenced from:
SignalRClient.(Inflater in _0DEB592A83220160D0A867EC9A8F696A).init(windowBits: Swift.Int) -> SignalRClient.(Inflater in _0DEB592A83220160D0A867EC9A8F696A)? in SignalRClient.o
SignalRClient.(Deflater in _0DEB592A83220160D0A867EC9A8F696A).init(windowBits: Swift.Int, memLevel: Swift.Int) -> SignalRClient.(Deflater in _0DEB592A83220160D0A867EC9A8F696A)? in SignalRClient.o
"deflateInit2", referenced from:
SignalRClient.(Deflater in _0DEB592A83220160D0A867EC9A8F696A).init(windowBits: Swift.Int, memLevel: Swift.Int) -> SignalRClient.(Deflater in _0DEB592A83220160D0A867EC9A8F696A)? in SignalRClient.o
"_inflateEnd", referenced from:
SignalRClient.(Inflater in _0DEB592A83220160D0A867EC9A8F696A).deinit in SignalRClient.o
"_deflateEnd", referenced from:
SignalRClient.(Deflater in _0DEB592A83220160D0A867EC9A8F696A).deinit in SignalRClient.o
"inflateInit2", referenced from:
SignalRClient.(Inflater in _0DEB592A83220160D0A867EC9A8F696A).init(windowBits: Swift.Int) -> SignalRClient.(Inflater in _0DEB592A83220160D0A867EC9A8F696A)? in SignalRClient.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Thank you for your help

You seem to be missing a reference to the libz library. README contains more details.

thank you very much moozzyk it fixed my isuue

https://i.stack.imgur.com/VCCrh.png
then build phases
link library


Transform row-oriented table to columnar-oriented table

Apache Doris has not supported row-oriented table any more.
In future release, row-oriented table should be transformed to columnar-oriented table.
Now we provide a schema change operator to accomplish this transformation.
Also we provide a script to generate all command about transformation, it should be performed before upgrade.
Script is stored in docs/row_to_column/ directory.

checks通过了


unable to debug

What are you trying to achieve?
I am tring to debug the codeceptjs app
What do you get instead?
After I set the launch.json with proprty
{
"type": "node",
"request": "launch",
"name": "codeceptjs",
"args": ["run", "--grep", "@your_test_tag"],
"program": "${workspaceFolder}/node_modules/.bin/codeceptjs"
}
I am getting error from vsc cannot laounth program {{my path}} settings the outfile attribute might help.

Provide console output if related. Use --verbose mode for more details.

# paste output here

Provide test source code if related

// paste test

Details

CodeceptJS version:
NodeJS Version: v8.11.4
Operating System:
Protractor || WebDriverIO || Nightmare version (if related)
Configuration file:

# paste suite config here

the path should be for vsc:
"program": "${workspaceFolder}/node_modules/codeceptjs/bin/codecept.js"


make nlp work for individual dictionaries

right now it only works with dataframes.
we'll probably want to move this whole block into fit:
                    col_names = self.text_columns[key].get_feature_names()

                    # Make weird characters play nice, or just ignore them :)
                    for idx, word in enumerate(col_names):
                        try:
                            col_names[idx] = str(word)
                        except:
                            col_names[idx] = 'non_ascii_word_' + str(idx)

                    col_names = ['nlp_' + key + '_' + str(word) for word in col_names]

done


MAINT: Fix uninitialized warnings in interpolate and cluster

Reference issue

Reference: https://github.com/rgommers/scipy/issues/42

Additional information

cc @rgommers

The PR title still says WIP,

Ohh, sorry. Forgot to change it after I added some more commits. Both if these modules are fixed

Great, in it goes - thanks Smit!


config bpftrace 0.10.0 version has problem

I built under  adeb debian env on Android device:
root@localhost:/bpftrace/build# cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang-7 -DCMAKE_CXX_COMPILER=clang++-7  -DBUILD_TESTING=OFF
-- The C compiler identification is Clang 7.0.1
-- The CXX compiler identification is Clang 7.0.1
-- Check for working C compiler: /usr/bin/clang-7
-- Check for working C compiler: /usr/bin/clang-7 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/clang++-7
-- Check for working CXX compiler: /usr/bin/clang++-7 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found LibBcc: /usr/lib/aarch64-linux-gnu/libbcc.so
-- Found LibElf: /usr/lib/aarch64-linux-gnu/libelf.so
-- Performing Test ELF_GETSHDRSTRNDX
-- Performing Test ELF_GETSHDRSTRNDX - Success
-- Found BISON: /usr/bin/bison (found version "3.3.2")
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- Looking for name_to_handle_at
-- Looking for name_to_handle_at - found
-- Looking for bcc_prog_load
-- Looking for bcc_prog_load - not found
-- Looking for bcc_create_map
-- Looking for bcc_create_map - not found
-- Looking for bcc_elf_foreach_sym
-- Looking for bcc_elf_foreach_sym - found
-- Looking for bpf_attach_kfunc
-- Looking for bpf_attach_kfunc - not found
-- Performing Test HAVE_BPF_PROG_TYPE_TRACING
-- Performing Test HAVE_BPF_PROG_TYPE_TRACING - Failed
-- Please install the libbpf development package (missing: LIBBPF_LIBRARIES LIBBPF_INCLUDE_DIRS)
-- Found LibBfd: /usr/lib/aarch64-linux-gnu/libbfd.so
-- Found LibOpcodes: /usr/lib/aarch64-linux-gnu/libopcodes.so
-- Performing Test LIBBFD_DISASM_FOUR_ARGS_SIGNATURE
-- Performing Test LIBBFD_DISASM_FOUR_ARGS_SIGNATURE - Success
-- Looking for sys/sdt.h
-- Looking for sys/sdt.h - found
-- Configuring done
-- Generating done
-- Build files have been written to: /bpftrace/build

build has no error, but bpftrace --info has abnormal message:
root@localhost:/bpftrace/build# bpftrace --info
System
OS: Linux 4.14.117-perf-gb69f97f #1 SMP PREEMPT Thu May 28 16:27:40 CST 2020
Arch: aarch64
Build
version: v0.10.0
LLVM: 7
foreach_sym: yes
unsafe uprobe: no
btf: no
bfd: yes
Segmentation fault

Can you post a stack trace of the segfault? Never seen this before.

sorry the segfault error is my local error.


Transfer PointNeXt to different problem

Hey there,
I really love the project, thanks!
I want to transfer PointNext or some other model to a different problem, where the task is to classify (or rather detect) a number of specific points in a point cloud. So to say somewhat of a keypoint detection in a point cloud. The dataset is rather small and it might not be as easy to generate new training samples.
I have a couple of questions:

What approach do you feel would be most likely to achieve that goal?
a. Transfer learning from the pretrained model for (I guess?) ShapeNetPartSeg?
b. Train a model from scratch?
c. Use a different algorithm family than the point cloud algorithms proposed here (because there might be better algorithms to solve such a problem?)

Can I model that problem in your openpoints framework? What I mean is can I create a yaml which will represent that kind of training (as decided in question 1)?

And how much benefit can I expect from generating more training samples?

To my understanding right now the ShapeNet input data contains of a 7dimensional vector. 3 position, 3 normal and 1...?. Part label? And could this be adapted to be only position and label?

Best regards

you can create more data using public avaiable dataset such as ShapeNetPart and S3DIS. Then transfer to your downstream.
yes
you have to do your own research
yes. play with the code on your self and you will know what to do


Notify in documentation useLinkedHeadings setting change after changes from #5225

@davisagli I know that we had these changes https://github.com/plone/volto/pull/5225 due to https://github.com/plone/volto/issues/5174 but it rubs me the wrong way that we make an assumption about this feature being useful only for logged-in users. If you want to enable for everyone you need to customize the render file just to remove the token condition.
What do you think about another option skipLinkedHeadlinesForAnon set to true by default?


ENH: wrap StreamingImageFilter for vector images

Re-creation of #1427 because I could not reopen it.

The test failure is unrelated. I think the Python build simply timed out.


Implement stream responses

Possible implementation
https://levelup.gitconnected.com/how-to-stream-real-time-openai-api-responses-next-js-13-2-gpt-3-5-turbo-and-edge-functions-378fea4dadbd

With this implementation, I'll swap 3.5 with 4

Typo on the merge message.


Feat/336 non attributable emissions page

Card: https://github.com/bcgov/cas-reporting/issues/336
Changes:-

Added Non Attributable Emissions Page: New page for collecting non attributable emissions data.

Create CheckboxGroup Widget

Modified NestedArrayFieldTemplate

Schema and UI Schema: Defined schemas to validate and structure the new data.

APIs/Services:

Get Gas type
Get Emission Category
Save Non Attributable Emissions Data
Get Non Attributable Emissions Data saved data for review.

Model for Non Attributable Emissions Data: Created a model for data storage and retrieval.

@dleard  Addessed all the requested changes

@dleard  Added Facility report id as wel to the model


build_material_checkbox.canvaskit.drawFrameDuration.average regressed

web_benchmarks_canvaskit build_material_checkbox.canvaskit.drawFrameDuration.average jumped from ~13ms to ~19ms.
See https://flutter-flutter-perf.skia.org/e/?begin=1597682768&end=1597696717&keys=Xf6596c634a675eea7960c0bf5ca0c829&xbaroffset=21088
https://github.com/flutter/flutter/commit/579cab8a868540b107e4e0c9f7afec3fbae11b60 is the culprit.

@LongCatIsLooong

@liyuqian revert?

Going to land the revert if I still need more time to investigate by the time the tree turns green.

Sounds good. Thanks for the quick resopnse!

Ran the benchmark (as instructed in the README file, flutter run --dart-define=FLUTTER_WEB_USE_SKIA=true --profile -d web-server lib/web_benchmarks.dart) 6 times in incognito mode for each of the scenarios below. Taking the median:

with Revert (#64062)
with the Revert but use a global key instead
without Revert (#64062) run 1
without Revert (#64062) run 2

14474 μs
14768 μs
15890 μs
15438 μs

On my PC it seems the patch introduced a ~6% - 10% time increase. @liyuqian does the approach sound reasonable?

"with the Revert but use a global key instead" means to add a global key to the mouse region to ensure the FocusableActionDetector's child doesn't have to be inflated again when we have to add/remove an Action or Shortcuts widget from the tree, i.e.:
  final GlobalKey mouseRegionKey = GlobalKey();

  @override
  Widget build(BuildContext context) {
    Widget child = MouseRegion(
      key: mouseRegionKey,
      onEnter: _handleMouseEnter,
      onExit: _handleMouseExit,
      cursor: widget.mouseCursor,
      child: Focus(
        focusNode: widget.focusNode,
        autofocus: widget.autofocus,
        canRequestFocus: _canRequestFocus,
        onFocusChange: _handleFocusChange,
        child: widget.child,
      ),
    );
    if (widget.enabled && widget.actions != null && widget.actions.isNotEmpty) {
      child = Actions(actions: widget.actions, child: child);
    }
    if (widget.enabled && widget.shortcuts != null && widget.shortcuts.isNotEmpty) {
      child = Shortcuts(shortcuts: widget.shortcuts, child: child);
    }
    return child;
  }

It seems this outperforms the original PR in the benchmark (defined in https://github.com/flutter/flutter/blob/master/dev/benchmarks/macrobenchmarks/lib/src/web/bench_build_material_checkbox.dart, toggling a grid of CheckBoxes on and off), as it is the simplest use case and the tree never changes height (as is the most common I think). But I'd expect it to become worse when the tree height does change.
I'm not entirely sure using global keys is the way to go, as there might be use cases that require constantly changing the height of FocusableActionDetector's subtree. @gspencergoog can I get your opinions on this?

On my PC it seems the patch introduced a ~6% - 10% time increase. @liyuqian does the approach sound reasonable?

Yes, this sounds reasonable. The devicelab has a bigger difference probably because it has a weaker linux box.

@LongCatIsLooong I think this may be why I implemented it the way I did originally, so using the GlobalKey is probably a good compromise.

@liyuqian the benchmark seems to be back to normal after the revert and didn't regress after the reland: https://flutter-flutter-perf.skia.org/e/?begin=1598045636&end=1598063764&keys=Xf6596c634a675eea7960c0bf5ca0c829&numCommits=90&xbaroffset=21088. Close?

Yes, closing now. Thanks for the fix!


Full PCB stack model

Things to note:

Parts have been sourced from vendors as per the links given.
Upper standoff 22mm did not have a freely available CAD file. So a slightly smaller one was used.
Stepper drive expansion board is merely a stand-in. No models or dimensions are available.

Closes #1020

Addressing some previous comments made on Slack:

These should be screws rather than nuts

That was not a nut, but a protruding standoff. It has not been fixed and screws have been added instead.

I see embedded html third party importation manifests in some of the models, those shouldn't be there

Removed

I would also go for slightly more subtle colors. These are too "pure RGB"
looks a bit like an 80s PC game on EGA graphics setting :smile:
looks like it's body 401 for the PCB board itself
how is there not a a "PCB green" appearance in the library?

Those were the colors that came with the vendor models and the PCB model supplied. Nevertheless, I have altered them to make them more subtle.

distances look about right, except interim board sits a bit taller -- board top to PCB plane distance is about 25mm

I'm not sure how I can change that distance. The pins mate in that configuration and leave that much space.

I don't really see anything to look at in terms of stepper driver. If it's just a gray rectangular slab then no that is not cool. Doesn't have to be every circuit on the board, but it should not be difficult to make the model, it's just:
pins
2 big capacitors
screw-down terminals
The background can be a decal of a photo we take of the board. I can do that for you, and I can take measurements of where everything is if you can't find drawings online.

I have created a likening of the stepper driver expansion board with these suggested additions.

First of all, you have to rebase this, because it is now showing 384 files changed. There is no way for me to reasonably follow the diff

Not sure why. That might have happened because I was asked to branch off of another issue and not master. I see that my commits have around 15 files changed. Anyways, it seems that branch was merged into master and I have rebased off of master, so it should be fine now.

You still did not address my comment about third party components not being in their correct directory.

Parts have now been moved. Although I've left PCB and the interim board in the same directory because I believe we designed them. Don't know if they need to be moved as well.

When loading the "full stack" assembly I get asked for this:

Not sure why. I don't get any such errors. The parts I have used are from the content center itself.

As to your comment about incorrect spacing, there are many ways to "make it right". A mate constraint has an offset parameter that is super obvious. You could also manually extrude the standoffs or terminals in question.

I know how to offset mates. That is not what I meant. I had applied the mate such that the pins interim board pins go fully into the nucleo. And I was informed that that was the only way it engaged with the nucleo. No standoffs etc. were used. That resulted in the spacing which was deemed "incorrect". Anyways, I have now made it 25mm.

This thing here does not make sense:

It has been mentioned in the model tree that it is a HDMI clearance stand-in. Since we don't have the model of the actual screen with all its parts, a stand-in was used to show that an HDMI port will sit in that area and that it isn't otherwise available. Would you like it removed?

display_panel_assembly.iam still not updated and, in fact, still broken because of the duplicate touchscreen files, which I also mentioned before.

I have renamed the newly created touchscreen file with bosses. Not sure which one you want to keep.

Just to clarify -- things that are not third party are things that one is expected to be mechanically assembling rather than receiving from a PCB contract manufacturer. So that means stand-offs that one is expected to install to mount the boards are part of the custom assembly. These assemblies should be in the bespoke directory.

They are not part of the mechanical design. Not fundamentally. They are not fully described as custom components here with all information necessary to manufacture them. They are only stand-ins for the purpose of having a geometrically true assembly. So, yes, they are all third party.

All stuff now in third party except PCB stack and display assembly.

Why are they from the content center? Wasn't there a part specified to you by Edwin? Why has that not been added to the BOM and then the CAD model from supplier used?

No. Screw specifications were not provided. Only that it should thread into the hex standoffs for a couple threads. So I selected from Content Center accordingly. I can update with manufacturer parts.

"I was informed" -- where and by whom?

I discussed with Edwin on a phone call.

It's not that I could not identify it. I said it does not make sense. Why was there a need for an HDMI clearance stand-in? It doesn't look like there are clearance issues here, but if you were thinking ahead to a model of an actual HDMI plug for later adding wiring to the CAD, why not add an actual HDMI plug model? I'm sure PCB people can tell you where to get one. Or at the very least have it look roughly the size of an HDMI port, because this thing looks totally not from this reality.

I have removed it. You're right. We are not going for a fully faithful version of the display anyway.

Why would we need 2 different ones? And again, why is the display panel assembly not updated with the whole PCB stack? Isn't this the whole point of adding these models?

Deleted the old 'display_assembly.iam'. Now the display has been integrated with the PCB stack assembly.

Latest commit addresses:

Hex socket screws changed from flat to rounded head
Updated readme with Digikey info for standoffs
Renamed touchscreen_with_bosses.ipt to touchscreen.ipt


Move to new attachment upload endpoint

Previously attachments were uploaded as a multipart form request when
creating the radar. Now they are uploaded using a separate PUT request
with the ID of the newly created radar.
One ambiguity here is how to handle attachment upload failure, since at
this point the radar is already created. Currently this is handled as a
failure of the entire radar creation, but that may need to change later.

Closes https://github.com/br1sk/Sonar/issues/21


Work conserve schedule

Add a work_conserving flag for _SynthesizedAutomaton which ensures that the scheduler always uses the full capacity of each scheduling slot.

TODO: reflect the changes in the documentation

I have reverted the changes to the API. Once this passes the tests, it should be ready to merge.

Looks great, thanks!


Allow passing in of variables to octopus deployment

https://github.com/vistaprint/octopus-jenkins-plugin/issues/5

Added a text area to put the properties to pass along
Added Variable API object
Added logic to merge interface values and default values from Octopus
Used .../OctopusDeploy-Api/wiki/DeploymentPreviews to get current set of variables to pass to deployment, and .../OctopusDeploy-Api/wiki/Deployments to figure out the need to pass FormValues with the variables defined as "[guid]": "value"

This is probably the begining since for now I am assuming all variable are strings.

Sorry for the delay on reviewing your PR. I will begin reviewing it July 19th!

Thanks for the plugin!  Passing variables really needed and appreciated.

Great thanks. It's possible that there will be more PR's as I work through other issues.


[Accessibility]  Focus is provided for non interactive section of dialog, tabbing further from button focus in iframe does not go to 'Close' button

Take jira cloud app as example as it has a similar UI with our app.

Focus is provided for non interactive section of dialog. That is the DOM of whole iframe content:

Tab order:
Expected:
1.Focus on the "Review Permission Button".

2.Press Tab. Then the focus should be provided for close button.
Actual:
1.Focus on the "Review Permission Button".
2.Press shift tab twice. Then the focus is provided for close button.

@qjkobe -Thanks for reporting your issue.
We will check this at our end and will get back to you.


make it easy to import lua modules for plugin authors

Currently it is difficult to require lua modules for a plugin and requires changing package.path. Neovim makes this easy by modifying the package.path so the plugin authors don't need to worry about managing that path.
Here is how it looks like before this change.
let s:is_nvim = has('nvim')
let s:has_lua = has('nvim-0.5.0') || (has('lua') && has('patch-8.2.0782')) " https://github.com/vim/vim/pull/6063

if s:has_lua
    if !s:is_nvim
        " make vim behave like neovim
        let s:plugin_dir = fnamemodify(expand('<sfile>:p:h') . '/../', ':p:h:gs?\\?/?')
        lua package.path = vim.eval("s:plugin_dir") .. '/lua/?.lua;' .. vim.eval("s:plugin_dir") .. '/lua/?/init.lua;' .. package.path
    endif
    lua asyncomplete = require('asyncomplete')
    lua asyncomplete.init()
else
    " TODO: fallback to vimscript so it works when lua is not available
endif

After this change it simplifies the code.
let s:has_lua = has('nvim-0.5.0') || (has('lua') && has('patch-8.2.0782')) " https://github.com/vim/vim/pull/6063

if s:has_lua
    lua asyncomplete = require('asyncomplete')
    lua asyncomplete.init()
else
    " TODO: fallback to vimscript so it works when lua is not available
endif

I can then add my lua files as follows for a plugin
asyncomplete.vim
  autoload
    asyncomplete.vim
  lua
    asyncomplete
      init.lua
      utils.lua
  plugin
    asyncomplete.vim

This allows me to just call require('asyncomplete') or require('asyncomplete/utils'). As a lua plugin author this simplifies requiring modules a lot.
I also updated #define scripts to include \n so it aligns with how it looks like. LUA_VIM_UPDATE_PACKAGE_PATHS comes from Neovim.

Codecov Report

Merging #6098 into master will increase coverage by 6.19%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #6098      +/-   ##
==========================================
+ Coverage   81.08%   87.27%   +6.19%
==========================================
  Files         141      142       +1
  Lines      155318   156934    +1616
==========================================
+ Hits       125935   136962   +11027
+ Misses      29383    19972    -9411

Impacted Files
Coverage Δ

src/if_lua.c
87.56% <100.00%> (+0.91%)
:arrow_up:

src/if_ruby.c
89.55% <0.00%> (-0.35%)
:arrow_down:

src/libvterm/t/harness.c
88.66% <0.00%> (ø)

src/xxd/xxd.c
74.93% <0.00%> (+0.20%)
:arrow_up:

src/drawline.c
84.78% <0.00%> (+0.47%)
:arrow_up:

src/if_tcl.c
86.67% <0.00%> (+0.54%)
:arrow_up:

src/optionstr.c
93.78% <0.00%> (+0.97%)
:arrow_up:

src/if_py_both.h
79.32% <0.00%> (+1.02%)
:arrow_up:

src/bufwrite.c
69.02% <0.00%> (+1.03%)
:arrow_up:

... and 129 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update deb1745...292b6e7. Read the comment docs.

Is there anything that would no longer work when including this?
Would be nice to have a test to check that it works.  You mention a few examples, perhaps those can be used?

@brammool added tests so should be good to review now.
There is one open issue which I wasn't able to find how to solve it since I'm not very familiar with the codebase. When eversomeone updates runtimepath we do need to call vim._update_package_paths(). Here is a pseudo code. Do you know where is the right place for this?
void on_update_runtimepath() {
    if has_lua() && has_lua_loaded() {
        lua vim._update_package_paths();
    }
}

On a side note is there a way to watch of variables getting changed? Whether it is variables such as runtimepath or global/buffer variables?

@brammool added tests so should be good to review now.
There is one open issue which I wasn't able to find how to solve it since I'm not very familiar with the codebase. When eversomeone updates runtimepath we do need to call vim._update_package_paths(). Here is a pseudo code. Do you know where is the right place for this?
void on_update_runtimepath() {
    if has_lua() && has_lua_loaded() {
        lua vim._update_package_paths();
    }
}

On a side note is there a way to watch of variables getting changed?
Whether it is variables such as runtimepath or global/buffer
variables?

Check out did_set_string_option().
--
You're as much use as a condom machine at the Vatican.
-- Rimmer to Holly in Red Dwarf 'Queeg'
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

@brammool thanks for the quick reply. Updated the did_set_string_option to call lua and update the package.paths. Tests seems to now work. This should be good to merge now.
As for your previous comments on this.

Is there anything that would no longer work when including this?

I don't see this being an issue. There are hardly any lua plugins for both vim and neovim. It is also extermely difficult to write lua plugins due to lack of docs which I do plan to improve in once I write more lua plugins. Neovim has added lot of docs on lua recently. Even for neovim I'm only seeing the rise of lua plugins in the past few months so having the behavior which is similar to neovim would be better. Currently the only thing I see missing from neovim that is explicit must is the lack of passing callbacks as I had mentioned in vim.fn and vim.call PR. So If you could add that support it would be great. Not familiar with lua c api and vim source code so not making good progress here on my own.

Please fix the test failure.

Codecov Report

Merging #6098 into master will increase coverage by 0.36%.
The diff coverage is 84.61%.

@@            Coverage Diff             @@
##           master    #6098      +/-   ##
==========================================
+ Coverage   86.96%   87.32%   +0.36%
==========================================
  Files         141      142       +1
  Lines      157155   157203      +48
==========================================
+ Hits       136665   137273     +608
+ Misses      20490    19930     -560

Impacted Files
Coverage Δ

src/if_lua.c
87.43% <81.81%> (-0.07%)
:arrow_down:

src/optionstr.c
93.79% <100.00%> (+0.20%)
:arrow_up:

src/vim9compile.c
92.98% <0.00%> (-0.67%)
:arrow_down:

src/vim9execute.c
93.51% <0.00%> (-0.57%)
:arrow_down:

src/vim9script.c
95.19% <0.00%> (-0.55%)
:arrow_down:

src/getchar.c
85.46% <0.00%> (-0.36%)
:arrow_down:

src/evalvars.c
95.06% <0.00%> (-0.17%)
:arrow_down:

src/eval.c
96.73% <0.00%> (-0.16%)
:arrow_down:

src/userfunc.c
93.03% <0.00%> (-0.07%)
:arrow_down:

src/channel.c
89.86% <0.00%> (-0.03%)
:arrow_down:

... and 57 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9cdcd1d...7aae42b. Read the comment docs.

Seems like it was broken here. I have fixed it to use split from lua instead of using vim's split which retruns user data.
Failures:

	From test_lua.vim:

	Found errors in Test_update_package_paths():

	Caught exception in Test_update_package_paths(): Vim(set):[string "local last_vim_paths = {}..."]:33: bad argument #1 to 'ipairs' (table expected, got userdata) @ function RunTheTest[39]..Test_update_package_paths, line 1


Move notebooks out from the docs folder

Example notebooks currently live inside the docs folder in not the most intuitive location.  We might consider moving these out (i.e. alongside the other examples).

Move these to a Project Pythia Cookbook

Keeping these notebooks through October 9th for the ECMWF conference.


Add HACL* Resolver

rust-crypto crate has been out of maintenance for years. I think snow should move to a more active cryptography library, such as ring, libsodium.
HACL* is a good choice and I try to add a HACL* backend for snow.
:)

Oh awesome! Thanks so much for the PR - I've been meaning to try out your crate in snow especially for wireguard-rs.
Yeah, the end goal is to drop rust-crypto all-together since it's unmaintained and relatively slow. Unfortunately, ring doesn't provide all the crypto suites we need for a full-ish Noise implementation, hence the RingResolver existing but not being able to fully replace our use of rust-crypto (the same goes for HACL* for now at least).
It would be worth talking about enabling ring and HACL* implementations by default, though, since they're possibly smarter choices than their pure-rust alternatives in rust-crypto...

The ring actually supports most noise required cipher suite, but it does not allow x25519 to use static keys. :(
I see you have a dalek branch and I think it's a nice replacement. :)

Nice, diff looks good and is feature-gated so I don't mind pushing this out without too much hubbub. I'll edit the README to clarify its existence.
I'll squash your commits and rebase them. Thanks @quininer, this is a good step towards eventually replacing rust-crypto as the default provider!


AutoEncoder Graph Construction

Hi Rajarshee Mitra,
I have question regarding your code. I am confused how your code construct the connections between layers. Specifically, in line 63 of stacked_autoencoder.py, you called the run function for different layers . it seems for me that you construct n different auto-encoders instead of one stacked with n hidden layers.
Best regards,
Mohammad

Notice that the variable x is the output of current layer, and then is passed to the next layer as input data. So I think the code is ok.

I believe the autoencoder here is being trained via greedy layerwise unsupervised training, thus each layer is trained one-by-one.

Thanks, @dawnranger  and @ljvmiranda921 !
@akbari59 , that's how a stacked auto-encoder differs from a multi layered NN. I hope your confusion has gone!


Sync upstream

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.3 out of 9 committers have signed the CLA.:white_check_mark: JackyWoo:white_check_mark: antonio2368:white_check_mark: alexey-milovidov:x: greensky00:x: tianjian600526:x: chenrui333:x: byronhe:x: hkadayam:x: bisegniYou have signed the CLA already but the status is still pending? Let us recheck it.


Node Upgrade

Context
Complete Node upgrade

Pull request: Node upgrade #199
Syhstem wide regression testing on node upgrade completed on 10/26 by the entire team


Local ssd

Hi there,
referencing issue #152
This is a first draft for a pull request implementing support for local ssd's. Currently only supports adding one ssd with the fixed size (I have no clue how attaching multiple ssd's would work). I can't get the integration tests to work, but I created a local release and tested it succesfully. Altough we needed this pull request: cloudfoundry/garden-runc-release#129 to be solved.
Only supports nvme for now, but it should be easy to support scsi as well.
Hopefully, third time lucky regarding the cla..

I was just looking for this feature. Are you still interested in seeing it through @stigtermichiel ?

@gerhard yeah, we have run this in production since the PR was submitted without any issues :)

Thanks @simonjohansson!
@dpb587 is this on your radar? cc @tysen @evandbrown

Sorry @gerhard, totally missed your message!

@evandbrown @jfmyers9 ping pong

Monthly ping

We're going to take a deeper look a this in the next month or so, but there's a few things that can be done first.
Why are there so many docs changes baked into this PR? This repo isn't a good place to keep things like docs on deploying CF - they're maintained elsewhere and some of these docs already have stale references to things like trusty. It also looks like some of the docs are not at all gcp specific.  At the very least the docs changes should be pulled into a separate PR.
Also can you resolve the conflicts?

My apologies, that must have happened when I tried to merge the develop branch in my branch. I will cherrypick the commits.

@stigtermichiel Thanks!


cant find some files that the project requires

hello,
great work, going through the readme file i successfully set up the flask project, but running the app gave errors that hint that a file (public/js/lib/d3.js) is required for the project to run appropriately and is missing.
I am studying applications of neural nets in the field of finance, so your project can really help me.
Can you please provide the required file.
Thanks

It looks like the entire public/js/lib directory is missing, which is a problem. You should be aware that this project is sorely outdated.
I've just pushed some updates so that it should now have all of the dependencies you need after running npm install. The application should then be able to be built and started running npm start. I have not fully tested this, though.


need PropertyGrid component

What problem does this feature solve?
可以根据 json 自动生成属性表格
What does the proposed API look like?
https://github.com/mokeyish/ngx-property-grid


repeat last reward after teminate state in make_target()

This is a suggestion to make learning better.
In the papar,

Terminal nodes. AlphaZero stopped the search at tree nodes representing terminal states and used the terminal value provided by the simulator instead of the value produced by the network. MuZero does not give
special treatment to terminal nodes and always uses the value predicted by the network. Inside the tree, the
search can proceed past a terminal node - in this case the network is expected to always predict the same
value. This is achieved by treating terminal states as absorbing states during training

What is "the same value"?
In the current implemantation, after terminating game, values and rewards are always 0.
In this new implementation, values are 0, but rewards are last_reward value.
This way, when the node is expanded past the end state in MCTS, the reword is repeated, which makes the choice clearer and leads to good policy learning.

These are test results about connect4.
Orange(A) line is the result after applying PR-108 and PR-121.
Blue(B) line is the result after applying PR-108, PR-121, and this PR.
Both are in the process of learning, but MuZero_reward shows that B is more advanced in its learning.

Any reason why you don't set the values of post-terminal states to be the value of the terminal state instead of 0?

Any reason why you don't set the values of post-terminal states to be the value of the terminal state instead of 0?

Any reason why you don't set the values of post-terminal states to be the value of the terminal state instead of 0?

There is no particular reason.
I think it works well even if value is repeated.

Any reason why you don't set the values of post-terminal states to be the value of the terminal state instead of 0?

There is no particular reason.
I think it works well even if value is repeated.

Hi again @mokemokechicken ! Sorry for the late response.
I think those changes could have the unwanted effect of orienting Muzero to certain trajectories that are considered good trajectories from Muzero's current model whereas it could explore trajectories that are better.
As it affects absorbing states, if num_simulations is quite low, these changes won't have a huge impact on the training if Muzero explores trajectories without getting to the terminal states but this could introduce a much more important bias in the training with a higher number of simulations.

Hi, @ahainaut
Thank you for your response!
Your point is certainly true.
In fact, the estimated value in MCTS will be larger than the original value (the sum of the rewards to be earned in the future).
I think it tends to make learning a little better, but if you don't think this is appropriate, please close this PR.
Thank you for your consideration!

@mokemokechicken,
Thanks again for your interest in this project and the energy you put trying to make this implementation better !
About this PR, I understand why it seems beneficial on a game like connect4 with only one reward given for an episode. To be clear about why I think this could not work, as we used the reward predicted by Muzero in the MCTS, the sum of discounted rewards could be wrong (at least at the beginning of the learning process) because the reward predicted by Muzero will not be accurate (which could be more visible if you consider a game with a much more complicated reward function).
An other issue is that, for a game like cartpole where the agent receives a reward for each time step before the game is terminated, Muzero will struggle to predict the end of a game in the MCTS as the reward will be repeated even after terminal states.
This is the MCTS plot we obtain after applying PR-122 in cartpole after a training of 80 played games (you can have the same plot using the diagnose model option in the menu) :

The root of the MCTS is the red circle.
As you can see, the most explored trajectory is the one taking only action=1 which leads to a terminal state pretty soon in the real game (not in the MCTS).
So if it's okay with you I'll close this PR.

@ahainaut
Thanks for your detailed explanation!
I think I understand what you are saying.
It's true that in such a case, it likely to get stuck in wrong learning, and this project will no longer be muzero-general.
Thank you very much!


Filter news + search news

This repo is a really good example:
Approximately over 80% of the news are "stared" and "forked".
Would be nice to be able to filter these out ;).
A search for the news would be nice as well.

@fadils is this related to the filtering stuff you were working on?

This is just a feaure request for a filtering feature:
I only want to see pull requests/comments/new issues
but i don't want to see which people started staring a repo just now or forked it (it's not important for me to know that)

We'll have to see if there's a way to filter this via Github's API in the request, otherwise we'll have to filter it client side


[Wait for #2581] [ Tensor ] Add inifinity check in Tensor

In this PR
This PR adds an infinity value check in Tensor data.
. rename the hasNaN to isValid
. add infinity check in isValid Function and now it checks NaN and Inf
. modify to check the blas_avx and blas_neon
. modify graph and model check is_valid rather than has_nan
. add unittest of isValid Function
Self evaluation:

Build test:	 [X]Passed [ ]Failed [ ]Skipped
Run test:	 [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2584. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

closed by  #2663


Library data request: adding lemmatizers for NLP libraries

Expected Behavior
N/A
Actual Behavior
Lemmatizers from WordNet@NLTK, spaCy, Pattern@GenSim are not accessible, although these libraries are supported. This hinders the development of NLP within LEAN environment.
Potential Solution
Download the additional data from the libraries and store them in LEAN for access.
# WordNet@NLTK
import nltk
nltk.download("wordnet")

# spaCy
python -m spacy download en

Add support to Pattern library
Reproducing the Problem
# WordNet@NLTK
from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()      # no downloaded data package "wordnet"
lemmatizer.lemmatize("XXXXX")

# Parallel@GenSim
import gensim
gensim.utils.lemmatize("XXXXX")            # no library support "Pattern"

# spaCy
import spacy
nlp = spacy.load('en')         # no downloaded data package "en"
doc = nlp(u"XXXXX")
for token in doc:
    token.lemma

System Information

Checklist

[X] I have completely filled out this template
[X] I have confirmed that this issue exists on the current master branch
[X] I have confirmed that this is not a duplicate issue by searching issues

[X] I have provided detailed steps to reproduce the issue

Pattern@GenSim

Pattern only supports python 3.6 https://github.com/clips/pattern

import spacy
nlp = spacy.load('en')

It's no longer 'en' but en_core_web_md or en_core_web_sm


Use TryAdd to add services rather than Add

@HaoK

Hi @kichalla, I'm your friendly neighborhood Microsoft Open Technologies, Inc. Pull Request Bot (You can call me MSOTBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Kiran Challa). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla2.msopentech.com.

TTYL, MSOTBOT;

Updated. /cc: @HaoK

:shipit:

Commit: 1cadc54c3b3fca71fcabd90965d2dd76ad1ebaf0


calico-policy-only.yaml in cluster with existing flannel does not work from version 3.26

Context
I'm trying to install Calico into Kubernetes cluster with existing Flannel overlay network and pods.
I suppose the calico-policy-only.yaml file is the one manifest I should use. I was successful to to setup it this way for Calico <= 3.25, but it does not work with newer versions.
Am I doing something wrong? Should I use different manifest or is there a bug?
Expected Behavior
It should work.
Current Behavior
Calico pods, DNS pods and other pods on flannel network does not have network access.
Below is one error message from events.
kube-system                      8m39s                    Warning   FailedCreatePodSandBox    Pod/kube-dns-autoscaler-568cfc9fc5-tmsl5
Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kube-dns-autoscaler-568cfc9fc5-tmsl5_kube-system_6285919b-c434-4ced-be73-f23e6cb14e33_0(b0e3a8aaa54fbec7c434847ac80eed1611bc216446ddbc14916a901c957b5017):
error adding pod kube-system_kube-dns-autoscaler-568cfc9fc5-tmsl5 to CNI network "k8s-pod-network": plugin type="calico" failed (add): error getting ClusterInformation: connection is unauthorized: clusterinformations.crd.projectcalico.org "default" is forbidden: User "system:serviceaccount:kube-system:calico-cni-plugin" cannot get resource "clusterinformations" in API group "crd.projectcalico.org" at the cluster scope

I verified RBAC are set correctly.
kubectl auth can-i create serviceaccounts/calico-cni-plugin -n kube-system --subresource token --as "system:serviceaccount:kube-system:calico-node"
yes

I've just found that if I add some additonal changes from this PR to the calico-policy-only.yaml file, I'm able to make my setup work.
To make it work I had to add following. It seems that I had to modify both ClusterRoles, but not sure which resources / verbs are actually required.
### add into calico-node ClusterRole
- calico-node

### add into calico-cni-plugin ClusterRole
  - apiGroups: ["crd.projectcalico.org"]
    resources:
      - blockaffinities
      - ipamblocks
      - ipamhandles
      - clusterinformations
      - ippools
      - ipreservations
      - ipamconfigs
    verbs:
      - get
      - list
      - create
      - update
      - delete

Your Environment

Calico version 3.26.4 and 3.27.3
Kubernetes version 1.27.10

Hi @JosefRypacek ,
Do you have any special purpose for using flannel for networking, calico comes with native networking without the need to use flannel. You can see what networking option would suit you best in our documentation as well as migration guide from flannel to calico
For using calico policy only mode with flannel, it is not possible to install calico on a cluster that already has flannel installed. We provide a way to install flannel as well as calico in policy only mode in one manifest. Please see docs for more details

Thanks, I've seen other networking options and recommendation, but I'm trying to use OCI cloud - OKE Kubernetes cluster which comes with Flannel networking out of the box. I may investigate other options, but as calico-policy-only.yaml worked in version <= 3.25, I would like to keep this way for now.
I actually followed this OCI documentation: https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengsettingupcalico.htm#manualcalicoinstall
After my modification to ClusterRoles it works as expected. I can probably also use canal.yaml file and remove flannel pods from there, but I thought calico-policy-only.yaml is the better way.
Can you please recommend which way is better (or less bad) to install Calico for policy only into Cluster with existing Flannel overlay network and pods?

Hiya @JosefRypacek,
Thanks for the extra info.
So, while it sounds like Oracle has configured Flannel in this scenario to be compatible with Calico, keep in mind that we haven't tested it nor do we support OCI officially, so we can't confidently advocate its use, or make any guarantees that everything is working as intended. For further support in this configuration, you would need to ask Oracle support.
Officially, we can currently only advocate the use of Flannel+Calico as shown in the docs and canal.yaml file.
The YAML shown on Oracle docs, on the other hand, was not originally intended for use on that stack, and has not been used in our installation docs for a few years now.
Right now I believe we do not have any official method for installing policy-only Calico on a cluster with a pre-existing Flannel overlay. Part of the reason for this is that some configurations of Flannel are completely incompatible with how Calico enforces policy, e.g, Flannel bridges break Calico policy between pods on the same node.

Thanks for the info. I'll keep it in mind and be sure to test compatibility and required features until there is some better way.
I have compared canal.yaml and calico-policy-only.yaml and have one more question.
Is there any benefit using Typha for Canal (Calico for policies and Flannel for networking) in this configuration? I read the documentation related to workload and number of nodes, but it's not obvious how much it's important when not using Calico networking. I was a little bit surprised when I saw Typha enabled in calico-policy-only.yaml.

Using Typha is also beneficial in policy mode as well. Typha proxies node's accesses to API servers, and helps with scalability of a cluster. Even in policy mode, Calico nodes need to process many updates about workloads, policies, etc.
In the end, it's up to users to enable or disable Typha. However, we recommend using it.