Aksing about uploaded documents only works when I first ask the AI on a specific one

System Info
Hello!
I wrote a code that is very similar to this one
https://python.langchain.com/docs/integrations/toolkits/document_comparison_toolkit
Who can help?
No response
Information

[ ] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[X] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[X] Memory
[X] Agents / Agent Executors
[X] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
class DocumentInput(BaseModel):
    question: str = Field()

llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")

tools = []
files = [
    # https://abc.xyz/investor/static/pdf/2023Q1_alphabet_earnings_release.pdf
    {
        "name": "alphabet-earnings",
        "path": "/Users/harrisonchase/Downloads/2023Q1_alphabet_earnings_release.pdf",
    },
    # https://digitalassets.tesla.com/tesla-contents/image/upload/IR/TSLA-Q1-2023-Update
    {
        "name": "tesla-earnings",
        "path": "/Users/harrisonchase/Downloads/TSLA-Q1-2023-Update.pdf",
    },
]

for file in files:
    loader = PyPDFLoader(file["path"])
    pages = loader.load_and_split()
    text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
    docs = text_splitter.split_documents(pages)
    embeddings = OpenAIEmbeddings()
    retriever = FAISS.from_documents(docs, embeddings).as_retriever()
        # Wrap retrievers in a Tool
tools.append(
    Tool(
        args_schema=DocumentInput,
        name=file["name"],
        description=f"useful when you want to answer questions about {file['name']}",
        func=RetrievalQA.from_chain_type(llm=llm, retriever=retriever),
    )
)
llm = ChatOpenAI(
temperature=0,
model="gpt-3.5-turbo-0613",
)

agent = initialize_agent(
    agent=AgentType.OPENAI_FUNCTIONS,
    tools=tools,
    llm=llm,
    verbose=True,
)

agent({"input": "did alphabet or tesla have more revenue?"})

Expected behavior
It's working so far, but when I directly ask the AI to compare the documents mentioning them by name,
the AI does not know what I am talking about!
I first have to ask questions specificly about each document and seperatly mentioning their names, then the llm can compare them and answer my questions.
Any idea to solve this issue?
I would appreaciate any help!:)
cheers

Thank you yo much for the fast and very detailed explanation!
This was the puzzle I was searching for, I will check it out!👍

Can you give me an hint where to start first when using ConversationBufferMemory?
How to load data directly into the memory, probably with RedisChatMessageHistory?


Detected new spammers

Detected new spammers

Thanks for the contribution.
Can you tell me how you have identified theses domains as referer spam?
Even if they clearly look like spammy domains, before merging I need to be sure we don't accidentally add a valid domain (false positive) in the process.

My site's language is Turkish. But they are Russian and Ukrainian sites. And not my link in their sites.      My irrelevant and unpopular subpages hits from that subdomains and subpages.
Traffic isn't organic. And from same IP addresses.
The attacks in bulk at certain times.
I use goaccess analyzing tool. I think my diagnosis is correct. Sorry for  my basic  English.

And the most important thing: They are not sending requests for my images and js files.

All right, this is more than enough to justify adding them. :)


Move post submission warning panel to footer

What is the context of this PR?
This moves post submission warning panel from pre footer template to the main footer. Changed described on this Trello.
How to review
Check post submission thank you page.
Checklist

[ ] New static content marked up for translation
[ ] Newly defined schema content included in eq-translations repo

 Move post submission "leave this page or close your browser" warning into the footer (XS)

As discussed, all post_submission_blueprint need a flag added, as the endpoints are already protected, it can just be show_warning_message: True. I believe there is some rationalisation that can be done on all end points (hide_signout_button, show_warning_message, sign_out_url etc), but this is possibly out of scope for this story

As discussed, all post_submission_blueprint need a flag added, as the endpoints are already protected it can just be show_warning_message: True with no extra logic. I believe there is some rationalisation that can be done on all end points (hide_signout_button, show_warning_message, sign_out_url etc), but this is possibly out of scope for this story

I've added new show_warning flag (should it be show_warning_message?). Tests to feedback and confirmation email added too. Thanks!👍

Since this is not census specific, I think the flag needs to be added to the default thank you context also.

Added to default thank you page and renamed flag to what you proposed, thanks!


Search AMI id

Hi
I wish there was a convenient way to search for AMI id. I know that can be done somehow with the aws cli, but well as usual it's not convenient to say the least !
Best regards,

Could you elaborate?
$ awless list images --filter id=ami-123456


Double serialization

I was looking at your ClosureProcessFactory, and I think you may be double serializing. Doing $this->fnSerializer->serialize($closure) on L50 should return the serialized string, as defined in the Serializer's interface, and you should not need to call serialize() again on L55. Likewise, you will want to adjust your unserialization logic to only unserialize once.

thanks @jeremeamia


Q# User Guide (merger of Q# Language and Q. Dev. Techniques section)

Still in very rough draft stages. So far:

brought in the changes from PR #486

To do:

[ ] general editing
[ ] bring in the rest of the files from Q. Dev. Techniques that were deleted in #486
[ ] rework the pages with more concreteness and code examples
[ ] diagrams where they can be useful
[ ] issues #660: discoverability of adjoint and controlled info
[ ] issue #658 jupyter (and perhaps other) commands quick-ref

Purpose is to make the current QDK in-depth section coherent. Could get help from more people to input on the format and target audience of this documentation. What's the intent?

Acrolinx Scorecards
A minimum Acrolinx score of 80 is required.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology.

Article
Acrolinxscore
Lowestcategoryscore
Spellingissues
Scorecard
Processed

articles/TOC.yml
100
100
0
link
:white_check_mark:

articles/contributing/style-guide.md
80
58
4
link
:white_check_mark:

articles/relnotes/0.3-migration.md
78
60
1
link
:white_check_mark:

articles/relnotes/index.md
76
43
20
link
:white_check_mark:

articles/techniques/local-variables.md
78
38
1
link
:white_check_mark:

articles/user-guide/basics.md
79
56
0
link
:white_check_mark:

articles/user-guide/callables.md
80
61
0
link
:white_check_mark:

articles/user-guide/expressions.md
86
61
2
link
:white_check_mark:

articles/user-guide/file-structure.md
79
62
2
link
:white_check_mark:

articles/user-guide/index.md
97
85
0
link
:white_check_mark:

articles/user-guide/operations-functions.md
81
52
4
link
:white_check_mark:

articles/user-guide/putting-it-all-together.md
83
51
2
link
:white_check_mark:

articles/user-guide/statements.md
77
47
7
link
:white_check_mark:

articles/user-guide/testing-and-debugging.md
80
60
1
link
:white_check_mark:

articles/user-guide/toc.yml
100
100
0
link
:white_check_mark:

articles/user-guide/type-model.md
78
56
1
link
:white_check_mark:

articles/user-guide/types.md
79
62
0
link
:white_check_mark:

More information about Acrolinx

Install Acrolinx locally for VSCode
False positives or issues
Troubleshooting issues with Acrolinx

Docs Build status updates of commit 5ddc116:
:warning: Validation status: warnings

File
Status
Preview URL
Details

articles/relnotes/0.3-migration.md
:warning:Warning
View
Details

articles/relnotes/index.md
:warning:Warning
View
Details

articles/user-guide/basics.md
:warning:Warning
View
Details

articles/user-guide/callables.md
:warning:Warning
View
Details

articles/user-guide/expressions.md
:warning:Warning
View
Details

articles/user-guide/index.md
:warning:Warning
View
Details

articles/user-guide/statements.md
:warning:Warning
View
Details

articles/user-guide/type-model.md
:warning:Warning
View
Details

articles/user-guide/types.md
:warning:Warning
View
Details

articles/TOC.yml
:white_check_mark:Succeeded
View

articles/contributing/style-guide.md
:white_check_mark:Succeeded
View

articles/techniques/local-variables.md
:white_check_mark:Succeeded
View

articles/user-guide/file-structure.md
:white_check_mark:Succeeded
View

articles/user-guide/operations-functions.md
:white_check_mark:Succeeded
View

articles/user-guide/putting-it-all-together.md
:white_check_mark:Succeeded
View

articles/user-guide/testing-and-debugging.md
:white_check_mark:Succeeded
View

articles/user-guide/toc.yml
:white_check_mark:Succeeded

articles/relnotes/0.3-migration.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.intro".

articles/relnotes/index.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.expressions".

articles/user-guide/basics.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/callables.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/expressions.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/index.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/statements.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.operations-functions".

articles/user-guide/type-model.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.expressions".

articles/user-guide/types.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Acrolinx Scorecards
A minimum Acrolinx score of 80 is required.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology.

Article
Acrolinxscore
Lowestcategoryscore
Spellingissues
Scorecard
Processed

articles/TOC.yml
100
100
0
link
:white_check_mark:

articles/contributing/style-guide.md
80
58
4
link
:white_check_mark:

articles/libraries/machine-learning/load.md
82
42
1
link
:white_check_mark:

articles/relnotes/0.3-migration.md
78
60
1
link
:white_check_mark:

articles/relnotes/index.md
76
43
20
link
:white_check_mark:

articles/techniques/local-variables.md
78
38
1
link
:white_check_mark:

articles/user-guide/basics.md
79
56
0
link
:white_check_mark:

articles/user-guide/callables.md
80
61
0
link
:white_check_mark:

articles/user-guide/expressions.md
86
61
2
link
:white_check_mark:

articles/user-guide/file-structure.md
79
62
2
link
:white_check_mark:

articles/user-guide/index.md
97
85
0
link
:white_check_mark:

articles/user-guide/operations-functions.md
81
52
4
link
:white_check_mark:

articles/user-guide/putting-it-all-together.md
83
51
2
link
:white_check_mark:

articles/user-guide/statements.md
77
47
7
link
:white_check_mark:

articles/user-guide/testing-and-debugging.md
80
60
1
link
:white_check_mark:

articles/user-guide/toc.yml
100
100
0
link
:white_check_mark:

articles/user-guide/type-model.md
78
56
1
link
:white_check_mark:

articles/user-guide/types.md
79
62
0
link
:white_check_mark:

More information about Acrolinx

Install Acrolinx locally for VSCode
False positives or issues
Troubleshooting issues with Acrolinx

Docs Build status updates of commit 7d91782:
:warning: Validation status: warnings

File
Status
Preview URL
Details

articles/relnotes/0.3-migration.md
:warning:Warning
View
Details

articles/relnotes/index.md
:warning:Warning
View
Details

articles/user-guide/basics.md
:warning:Warning
View
Details

articles/user-guide/callables.md
:warning:Warning
View
Details

articles/user-guide/expressions.md
:warning:Warning
View
Details

articles/user-guide/index.md
:warning:Warning
View
Details

articles/user-guide/statements.md
:warning:Warning
View
Details

articles/user-guide/type-model.md
:warning:Warning
View
Details

articles/user-guide/types.md
:warning:Warning
View
Details

articles/TOC.yml
:white_check_mark:Succeeded
View

articles/contributing/style-guide.md
:white_check_mark:Succeeded
View

articles/libraries/machine-learning/load.md
:white_check_mark:Succeeded
View

articles/techniques/local-variables.md
:white_check_mark:Succeeded
View

articles/user-guide/file-structure.md
:white_check_mark:Succeeded
View

articles/user-guide/operations-functions.md
:white_check_mark:Succeeded
View

articles/user-guide/putting-it-all-together.md
:white_check_mark:Succeeded
View

articles/user-guide/testing-and-debugging.md
:white_check_mark:Succeeded
View

articles/user-guide/toc.yml
:white_check_mark:Succeeded

articles/relnotes/0.3-migration.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.intro".

articles/relnotes/index.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.expressions".

articles/user-guide/basics.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/callables.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/expressions.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/index.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

articles/user-guide/statements.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.operations-functions".

articles/user-guide/type-model.md

[Warning] 1 invalid cross reference(s) "microsoft.quantum.manual.expressions".

articles/user-guide/types.md

[Warning] Author Gillenhaal Beck cannot be recognized in GitHub.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Superseded by #743 , closing


Poll detail: Result lists can't show deleted users

Create an anonymous poll
Vote with a few accounts
Stop and publish poll
Delete one of the accounts who voted
Reload page
Take a look at the poll result lists => The entitled users table is broken

TODO:

[ ] Fix entitled ussers table
[ ] See if the votes table is also broken (i.e. when you use a different method than anonymous) and, if yes, fix it

Works for me

Seems to be fixed in current main


Give reasonable recommendations or at least consistent defaults for CLMBR vocabulary size

Is your feature request related to a problem? Please describe.
Various papers that use CLMBR-T-Base (eg https://arxiv.org/abs/2311.11483) use a vocabulary size of 65,536 codes.
The tutorial trains the tokenizer with a vocab_size of 128 but then sets up the CLMBR task with a clmbr_vocab_size of 64.
The FEMRTransformerConfig sets a default vocab_size of 32,768, but it's unclear if it's even possible to use that default through standard routes, as a lot of the methods that internall instantiate FEMRTransformerConfig have vocab size as a required argument with no defaults.
Describe the solution you'd like
Ideally both the tutorial and documentation would (A) provide guidance on how to choose the vocabulary size and (B) offer reasonable defaults for the scale of data we'd imagine most users are working with. At very least, the vocab size used throughout the tutorial should be consistent.

I do not really have any good guidance here. I will add a note that 128 is 100% wrong though and purely to make the tutorial run fast

Ok. The tutorials have been updated and I am going to close this issue. Thanks for bringing it to my attention. It was not obvious that 128 was a dummy value.


feat(result): support combining multiple results

Allows combining multiple results into one.
Error Message of failing results are concatenated.
CSharp Functional Extensions provide a similar feature: https://github.com/vkhorikov/CSharpFunctionalExtensions/blob/master/CSharpFunctionalExtensions/Result/Methods/Combine.cs#L18
Open questions
Currently, only ErrorMessages of type string are supported.
Do we want to invest some more time to aggregate Error-Objects as well?

Idea Maybe, we could improve .combine even more providing a better experience.
const user = Result.success({ name: 'Somebody' });
const order = Result.success({ name: 'Some Name' });

const result = Result.combine({ user, oder }).map(({ user, order }) => ... )
//                                                           ^ Result value combines values of all Results.

Added types and a few tests.

Nice readable tests! 👍

Introducing Record in favour of using tuples.
Reason: Accessing members of a Record is more readable.
// Current Implementation
Result.combine( result1, result2).map(results => results[0] ...)

// vs. Record

Result.combine({ result1, result2 }).map(results => results.result1)

Thanks, @KyleMcMaster, I updated the implementation.
Let me know if I can improve something.

Hi @seangwright & @KyleMcMaster,
the PR is ready for review. 👍


Changing "VMs" to "machines" in this sentence

The rest of this paragraph talks about nodes being "machines," whereas this sentence is more specific, referring to virtual machines ("VMs"). The article this sentence points to also uses "machines" and isn't specific to VMs.

/lgtm
/approve


default nonce dateNow, include nonce in computeResult params

Fixes # .
Changes proposed in this PR:

default getNonce to date.now if not in redis
include nonce in computeResult params as commented (The UTC timestamp, used to prevent replay attacks) in provider endpoint but not in used yet

there is already a PR opened #1386 that fixes this for computeResults. We decided to keep that as a draft and merge once we bump everything to use the latest version of contracts

there is already a PR opened #1386 that fixes this for computeResults. We decided to keep that as a draft and merge once we bump everything to use the latest version of contracts, till then we use version 1.0.0.alpha-26 in that pr so we can use the build locally in the market

alright, then i just abandon the PR


Organization Test and Test Data Refactor

This pull request refactors the backend organization unit tests for the merge, as well as separate mock data for the organization test and the mock data for the live demo of the page when run with honcho start.
Major Changes

Refactor organization tests into a new folder for better organization.
Ensure all organization tests pass by adding permissions to test data.
Separate test data into organization_demo_data for the live demo, which contains all organizations, and organization_test_data for the pytests.

Testing
These changes were thoroughly tested on the local server by running honcho start.

This PR resolves #50
This PR resolves #51

@atoneyd resolved - there are now two separate scripts to test respective data.


Relicense under dual MIT/Apache

It was suggested in #21 to relicense under dual MIT/Apache. I personally don't have any objections to this so I'm opening this issue to start the discussion. As far as I understand it I need signoff from all contributers of "creative works" to the project. It's not entirely clear what that constitutes but I'll be requiring signoffs from any contributors with over 100 lines changed. Any other contributors are free to voice their consent or objections as well, which I will take into account.
Required

[ ] @newpavlov
[ ] @matthiaskrgr
[x] @dgriffen

Optional

[ ] @willstott101
[ ] @sgodwincs

No optional objection from me

No objections.

No objections.

@matthiaskrgr Ping

@dgriffen Sorry for the delay, I somehow missed previous notice.
No objections.


Small changes to make it compatible with a whole new gold price API I…

… stood up
Description
Replace this with a description of your changes.
Copilot

🤖 Generated by Copilot at 5f4e8d0
Summary
🔄🦏🔑

This pull request improves the goldpriceticker app by switching to a more reliable and flexible API, adding a new metal option, and enabling user-provided API keys. It also secures the API key with encryption.

The old API is doomed, we need a new source of power
We summon the rhodium, the rarest of the metals
We enter our secret key, encrypted in the darkness
We run the goldpriceticker, we watch the prices rise

Walkthrough

Load and use secret.star file to decrypt API key for metals API (link)
Add rhodium as an option for the precious metal to display (link, link)
Update API endpoints to use new domain and authentication (link, link, link)
Add text field for API key in app configuration for testing purposes (link)

I believe the render is failing because it isn't using the API key when its run with pixlet.


openai.api_key in OpenAIEmbeddings  is unsafe  on  concurrency

System Info
like:
import openai
openai.api_key = openai_api_key
problem:
api_key is a global variable, it is not safe on concurrency  if we have different api_keys to be used concurrency.  api_key will out of control.
Who can help?
No response
Information

[ ] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
use local variable
Expected behavior
to be fix

you should move this issue to https://github.com/openai/openai-python


Unable to reproduce the results of the paper

For ImageNet_LT，I just use default config in the code, but cannot reproduce the results in paper Table3(a).

For stage1, my result is（some last logs when training complete）:
Epoch: [30/30] Step:   440  Minibatch_loss_performance: 2.833 Minibatch_accuracy_micro: 0.438
Epoch: [30/30] Step:   450  Minibatch_loss_performance: 2.886 Minibatch_accuracy_micro: 0.379
Phase: val
100%|██████████| 79/79 [01:40<00:00,  1.34it/s]
Phase: val
Evaluation_accuracy_micro_top1: 0.220
Averaged F-measure: 0.175
Many_shot_accuracy_top1: 0.427 Median_shot_accuracy_top1: 0.113 Low_shot_accuracy_top1: 0.007
Training Complete.
Best validation accuracy is 0.220 at epoch 30

Few/Low shot acc 0.7% is better with 0.4% Plain model in Table3(a) .

[Below is IMPORTANT!!!!!]
2.However for stage2, my result is（some last logs when training complete）:
Epoch: [60/60] Step:   440 Minibatch_loss_feature: 0.569 Minibatch_loss_performance: 2.938 Minibatch_accuracy_micro: 0.566
Epoch: [60/60] Step:   450 Minibatch_loss_feature: 0.567 Minibatch_loss_performance: 2.845 Minibatch_accuracy_micro: 0.539
Phase: val
100%|██████████| 79/79 [01:34<00:00,  1.02it/s]
Phase: val
Evaluation_accuracy_micro_top1: 0.340
Averaged F-measure: 0.324
Many_shot_accuracy_top1: 0.401 Median_shot_accuracy_top1: 0.334 Low_shot_accuracy_top1: 0.197
Training Complete.
Best validation accuracy is 0.341 at epoch 48
However Many, Median and Few/Low shot acc are 40.1%, 33.4% and 19.7%, which are a little diff with 43.2%, 35.1% and 18.5% in  "Ours" model in Table3(a) .
And I retrained for several times, the Many-shot acc always some lower than 43.2%.

Are there any tricks not released?

@GuohongLi Thank you very much for asking and sorry for the late reply. As replied in your last issue (https://github.com/zhmiao/OpenLongTailRecognition-OLTR/issues/50#issue-524159914), we finally debugged the published code and current open set performance is:
============
Phase: test
Evaluation_accuracy_micro_top1: 0.361
Averaged F-measure: 0.501
Many_shot_accuracy_top1: 0.442 Median_shot_accuracy_top1: 0.352 Low_shot_accuracy_top1: 0.175
==========
This is higher than we reported in the paper. We updated some of the modules with clone() method, and set use_fc in the first stage to False. These changes will lead us to the proper results. Please have a try. Thank you very much again.
For Places, the current config won't work either. The reason why we could not get the reported results is that we forget that on the first stage, we actually did not freeze the weights. We only freeze the weights on the second stage. We will update the corresponding code as soon as possible.

@GuohongLi Hello, we just updated new configuration files for places, and the newly implemented results are a little better than reported. Please check out the updates. Thanks!


If dataset has 10 files and you request 11, shoudl not cause us ot try to download dataset again

In short, pay attention to the partial file. :-)

By the very nature of how we work, this won't happen.


Build new schools page

Implement new schools page.

Rework map and introductory text layout
Adjust map so that it no longer recentres after a load. Avoids an unnecessary jump in the map position
Tweak the clustering at different zoom levels
Remove popup content from GeoJSON served to build map layer. Speeds up the map loading as we don't have to load and process all school configurations. Use school id for feature id
Add new route, controller and template for serving popup content
Adjust onEachFeature to dynamically load popup content
System specs to check all behaviour. Provides additional tests for the search component behaviour as well as its integration with the controller.

Some of the spec files are just changes to reduce dependency on navigating via the schools page to test school specific functionality. Seems unnecessary to keep testing that navigation in other specs.
I've not ended up changing the JS much to reduce use of JQuery. The map javascript is using JS fetch and promises rather than JQuery functionality.

As discussed will ask for user feedback and then refactor if needed to adjust the totals.


Cannot import CustomGoogleScholarTopPublicationCitations

I'm currently using a macOS Big Sur, python 3.11.1. While I can import CustomGoogleScholarOrganic without issues, I am unable to import CustomGoogleScholarTopPublicationCitations, with the relevant error message included below.
>>> from google_scholar_py import CustomGoogleScholarTopPublicationCitations
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'CustomGoogleScholarTopPublicationCitations' from 'google_scholar_py' (/Users/rohanjha/.pyenv/versions/3.11.1/lib/python3.11/site-packages/google_scholar_py/__init__.py)
>>> from google_scholar_py import CustomGoogleScholarOrganic
>>>

Any help on this would be greatly appreciated! Additionally, example_usage.py says that documentation exists for each of these functions. Could you please point me to the documentation if possible? I'm interested in scraping the citation counts for various papers, and would love to read up on the best way I can do so. Thanks for your support!

Hi, I'd read the thread linked above but I don't believe this is a duplicate because I'm using Python 3.11 and can only not import 1 package (while importing the other 2 works fine)

@rohanjha123 I apologize, it was a typo by me. There's no CustomGoogleScholarTopPublicationCitations class. I've updated example_usage.py
Class naming is confusing. This is something I'll update in the future.
Correct code example:
from google_scholar_py import CustomGoogleScholarTopPublicationArticle
import json

top_publication_citation = CustomGoogleScholarTopPublicationArticle().scrape_google_scholar_top_publication_articles(
    journal_publications_link='https://scholar.google.com/citations?hl=en&vq=en&view_op=list_hcore&venue=TdhLrHqKTh8J.2022',
    pagination=True,
    save_to_csv=False,
    save_to_json=False
)

print(json.dumps(top_publication_citation, indent=2, ensure_ascii=False))

Let me know if this works for you.


RuntimeError: can't start new thread

Hello, in just one cluster (out of ~20) GPM is not starting (Crashloopbackoff) with this log output.
We are using version 0.7.0 and deploy the helm chart with ARGOCD. Kubernetes Version is 1.24.13
Please let me know, if other infos are needed.
Any help appreciated. Thank you!
...
[2023-10-06 07:16:41 +0000] [8] [INFO] In cluster configuration loaded successfully.
[2023-10-06 07:16:41 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 95, in init_process
super().init_process()
File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/base.py", line 142, in init_process
self.run()
File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 214, in run
callback(key.fileobj)
File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 150, in on_client_socket_readable
self.enqueue_req(conn)
File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 117, in enqueue_req
fs = self.tpool.submit(self.handle, conn)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 176, in submit
self._adjust_thread_count()
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 199, in _adjust_thread_count
t.start()
File "/usr/local/lib/python3.11/threading.py", line 957, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread

Hi @Harald-koeln
This seems indeed strange, let me ask you some questions to better understand the issue:

Are the rest of the applications running without issues on the cluster?
Are the other clusters that you have running the same version of Kubernetes and GMP?
Do you have any resource limits set? can you share the values that you are using to install the chart? please make sure to mask all the sensible info like secrets and URLs
Is the node where GPM is running healthy?
are there any other security tools or measures running in the cluster that could prevent GPM for creating threads?

Thanks!

Hi @ralgozino ,
thanks for your reply. I will answer your questions, but actually I do not have the need anymore to deploy GPM on that cluster. I've figured out that trivy-operator was additionally deployed on that cluster by another team. We do not need both and maybe this is also the reason for the problems.
Concerning your questions:

yes, gatekeeper (and trivy and other applications) are running without issues
yes, there are all the same version
no resource limits set by me, only default helm chart values (see below)
node is healthy
trivy (probably responsible for GPM problems)

      source:         chart: gatekeeper-policy-manager         repoURL: https://sighupio.github.io/gatekeeper-policy-manager         targetRevision: 0.7.0         helm:           releaseName: gatekeeper-policy-manager           values: |-             config:               secretKey: "gatekeeper-policy-manager"             ingress:               enabled: true               hosts:                 - host: "gpm.{{ metadata.annotations.cluster_wildcard_domain }}"                   paths:                     - "/"

I'm glad you sorted it out, I'll probably do some tests with trivy-operator (I haven't heard of it before) anyway to see if there's something we can do to make them compatible.
thanks!

hey @Harald-koeln
I tried reproducing the error with some load testing but I can't trigger it.
Do you have some limit sets on the number of processes that a container can run? or are the used inodes in the node close to the limit maybe?
Anything particular of your setup that we should know to replicate the issue?
I wonder if the same/similar issue happens to you with the new Go backend that is in development, would you mind testing it? You just need to change the image tag to go, i.e. v1.0.7 -> go. Please let me know if it happens there also. Notice that the go backend does not support OIDC auth yet in case you are using it.

Hi @ralgozino , thank you very much. The go-Version is working on all 5 clusters where I observed problems with the python version.
As far as I know there are no special limits about the number of processes or inodes.
We do not use OIDC here so using the go-version is fine for us. Thanks!

glad to hear that! any feedback on the go backend version is very welcomed :-)


[Suggestion/Request] Specify the seeds used for features/structures

I wanted to suggest a feature/change. Vanilla uses a hardcoded seed, and this mod randomizes the seed instead. Could this mod provide an optional config that lets you specify what seeds to use for those, instead of randomizing them?

The mod needs to ensure seeds are kept consistent throughout reboots, so it saves and loads the seed values from world/feature-seeds.json and world/structure-seeds.json. If you want to manually specify the seeds, you can generate these files and use them in another world, with your adjusted values.


Conflict with gitlens plugin

Description
The plugin appears to have a conflict with the gitlens plugin.  When this plugin and the gitlens plugin are enabled, all the gitlens blame annotations show "cannot determine recent change or authors".

I am not sure this is a conflict.  Just ran some other tests and the same error appears when doxdocgen is disabled.


Create a project roadmap

Some of the features of RHPMAN were not implemented in this inital version as they were not required for the simulation study for this masters thesis.
However, a list of the missing features and the order they should be added to the project should be created.
This should be one by creating a project and a number of tickets that should be put in a prioritized order.

Ill probably leave this task for after github finishes their new projects feature since Im not a fan of the current table view.


Cannot hook some function on iOS 9.3

The ftruncate() function in AudioToolBox cannot be hooked on iOS 9.3.
On iOS 8.3 it works.

This cannot be fixed today now.
https://github.com/comex/substitute/issues/13
As @comex commented:
"comex commented on May 12, 2016
For the record, if you have a debugger active, the process is granted permission to run unsigned code - so if this is for testing, you may be able to use Substitute. If you need to do it in production... and the fact that you really shouldn't doesn't sway you... you'd have to find a way to sign modified versions of the OS code pages.
dyld options won't affect anything because the optimization is done when the dyld cache is built, which on iOS is done ahead of time by Apple as part of building the OS image."


Tobin tri test cleanup

Replaces #2240
Includes all of @krOoze clean-up work with a couple of minor additions to workaround specific device issues and a minor fix-up.
The CL from @krOoze has already been through lots of review and testing so unless there are strenuous objections to something there no need to comment on those changes. This is primarily to allow a review of my additions which I believe are minor but wanted to put them up for review before merging.

Great! I will rebase #1983 momentarily, and we shall see if we got anywhere with this... 😋

Hmm, you just remove the depth attachments. That kinda sets me back to square one. Besides, it kinda makes the three tests invalid. E.g. without depth attachment, the VkPipelineDepthStencilStateCreateInfo is ignored, therefore stencilTestEnable is actually disabled, which makes the test cases actually a valid usage.
I guess I need to rewrite it once more. The code still looks like it needs it anyway. The clear (which I assume is the driver problem) should theoretically not be necessary at all.

Yes, the clear is the problem. Let me re-test with the DS attachments back in for those 3 tests as I made that workaround before a couple of others and we also just updated the devices in our farm.


Break into modules

There will be 3 libraries:

[x] Thing.js: utilities for working with thing objects
[ ] Evironment.js: utilities for working with Evironment objects (which contain things and interrelationships between the things).
[x] Grow.js: a wrapper for Thing.js and Environment.js that connects to a Grow-IoT instance.

There are other good libraries out there for Javascript robotics, etc, but this will allow us to build some pretty cool control systems out of IoT devices I think...

Behold: https://github.com/CommonGarden/Thing.js
I'm writing tests and cleaning things up.

Environment.js will be coming soon.


Ci/improvements

This PR makes minor improvements to the build (Makefile.toml) and CI for better maintainability and to increase test stability in the CI.

size-limit report 📦

Path
Size

target/wasm32-unknown-unknown/release/cw_template.wasm
176.89 KB (0%)

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add support for compose-jb

Is your feature request related to a problem? Please describe.
I want to use this library with pager from accompanist in my Compose for Desktop app.
Describe the solution you'd like
Add support for compose-jb to be able to use this as a multiplatform library, the code looks to not depend on android only code so it should be possible.
Describe alternatives you've considered
If not, I can fork this and make a new version that supports Compose for Desktop.
Additional context
I have migrated all my android app from xml + fragments to compose only to make my app work for desktop as well and it works well, but I need a Pager and I can't use accompanist because it supports Android only.

Thanks for the report. Yeah this is something I've thought about, but it's actually pretty difficult to have a single build using the same source files, but with different plugins.
I tried having 2 modules (one JB, one Android) and then setting the source sets to use shared/src/blah/Snapper.kt, but Android Studio didn't like it. I think my next attempt will be to attach a copy task to Compose JB's compile step, to copy over the 'Android' source files to a temporary/gen location.

You shouldn't need to copy sources and use multiple plugins. Just by converting to a Kotlin Multiplatform project with Android support and using the org.jetbrains.compose Gradle plugin instead of the Android plugin's buildFeatures, the artifacts will be available for use by both Android and Desktop.
Here's the build.gradle.kts for a library I recently published that does exactly this. The repo also has example sub-projects that apply the normal Android/Compose or Desktop configurations, showing that the library can be published with JB compose, but used on Android without needing the Android app to be MPP/JB-compose.

@cjbrooks12 the jetbrains project uses an older version of compose, you will not be able to use this library on the latest compose version for Android, it's better to have 2 projects for each

I copied the library code into my compose-jb project to test it out, but snapping doesn't work. I'm pretty sure it's related to this issue, which means performFling() never gets called.

@zacharee yeah that looks like a hard stopper for this. The flinging implementation on Desktop looks like it completely bypasses all of the scrollable state calls.


Responsive Web Design for home page

@iwisunny Could you pls help review and merge this PR

@FeynmanZhou All right


Use BeautifulSoup instead of lxml for MJML/HTML parsing

Fixes #6
Fixes #10
Fixes #14
Fixes #19

Some of the upstream alignment tests were failing. I passed the source MJML back into mjml-cli locally and rebuilt the HTML, and I then updated the "-expected.html" files (some style tags were missing). All tests then passed.

That's quite an impressive pull request which could really fix a lot of issues, thank you so much. I didn't have time to really dig into this but I'll try to split my review into smaller pieces to understand what has changed and why.
Switching to Beautiful Soup might be a good solution to resolving the issues around the pretty strict lxml parsing and these issues are actually the single biggest core issue of this project right now so highly valuable.
I was wondering about failing upstream tests. So you are saying that after your Beautiful Soup migration some upstream tests were failing which did work before? And these tests started to pass once you regenerated the ...-expected.html files?

Which version of mjml (upstream) did you use?

I would love to structure the changes in a way that switching the parser does not require any changes to the tests.
What is your confidence in the correctness of your changes? Did you compare the HTML output manually against upstream HTML? Do you use the new code already in production? Once I got a chance to read up on your code I can plug it into a customer's test suite to ensure the new code did not change in unexpected ways.
Last but not least: Thank you for tackling this high-value issue. Looking forward to merging your code.

Looks like it was only the HTML for the "missing features" tests that needed to be regenerated (after I moved them alongside our other test data). I used MJML v4.12.0 to generate - the HTML. These tests were already failing previously, of course, so it's possible issues with the HTML flew under the radar.
Related, I wonder if there's some way we could call mjml-cli from our tests to generate the expected HTML on the fly instead. It would be nice too, because we could then ensure we were always matching the same version of MJML in all of our tests.

These tests were already failing previously, of course, so it's possible issues with the HTML flew under the radar.

That makes sense.

Related, I wonder if there's some way we could call mjml-cli from our tests to generate the expected HTML on the fly instead. It would be nice too, because we could then ensure we were always matching the same version of MJML in all of our tests.

Generating it on the fly could bring quite a few issues and makes the test suite more brittle but I have a script to regenerate all HTML files. I could add this to the git repo.

Ok, I went through your changes and I think they are fine. I noticed that the ...-expected.html in the missing_functionality were out of date so I updated them in a separate commit fb83576. Unfortunately github's web editor can not handle that.
Would you mind rebasing your commit on top of main + squashing everything into a single commit? Bonus points for explaining the motivation the rationale of that change (something about the tickets you mentioned in your initial comment) in the commit comment.
As a final validation I'd like to plug your changes into a customer's CI pipeline to see if anything breaks there. I'll do that on Monday. After that we're ready to merge :-)

@FelixSchwarz Alright - should be all set!

Everything looked fine in our CI pipeline so I think there should not be too many surprises :-)
One thing which might be affected is that BeautifulSoup is more leniant when it comes to XML syntax so maybe we should add some (optional) XSD validation to ensure the inputs not completely off-base.
With your changes now in main there are enough new features/bug fixes to warrant a release. I plan on doing that shortly once I completed a few tasks:

#24
#23
#22

Thank you very much, the BeautifulSoup changes much less intrusive than I expected and really solves a lot of issues I had with lxml.


[BUG] Unexpected behavior when time scale min/max doesn't cover entire range of values

See this: https://jsfiddle.net/moonchen/hvzaabxn/6/
Expected Behavior

In the example above, I expect a trapezoid shape graph that spans 6:00:01.3 pm to 6:00:01.5 pm.
Current Behavior

Currently, the graph starts at 6:00:01.050 pm, and ends at 6:00:01.950pm.  Notice the dark tick marks at the requested min/max values.  Also, the background fill extends outside the graph area.
Possible Solution

I can hook afterBuildTicks to set axis.min and axis.max to their proper values, and manually construct the ticks array.  This fixes the x-axis range, but doesn't fix the fill problem.
Steps to Reproduce (for bugs)

Graph a data set using a time scale on the x-axis.
Set the x-axis min or max to be inside the data's x range.

Context

I'm trying to draw a real-time graph that show time-series data.  This bug shows up when data history exceeds my graph window.  I want data history to exceed the graph window because I want the left-most line segment to be correct.
Environment

Chart.js version: 2.6.0
Browser name and version: Safari 10.1.2

I think some or all of these issues have been fixed in master. @simonbrunel and @benmccann have the details

Probably fixed by https://github.com/chartjs/Chart.js/pull/4522

Fixed by #4522 (http://jsfiddle.net/hvzaabxn/7/)


Replace eth-lightwallet with ethers.js

Problem Description
Currently we are using eth-lightwallet for wallet related functionality. But in the past we had some problems with it as the library is not very well maintained (last commit almost a year ago, a lot of open issues and PRs). There was recently the problem with outdated dependencies, the incompetence to distinguish two methods with the same name from the contract ABI and some more.
Proposed Solution
Replace it with alternative ethers.js library. We considered ethers.js also already in the past, but it lacked some key features for signing raw transactions and RLP encoding of transactions. The new version looks very promising as it supports those features and many more. It also comes with types for TypeScript and has interfaces for custom Providers and Signers (https://docs.ethers.io/ethers.js/html/index.html). The only feature that is missing is the encryption of arbitrary messages.
Why migrating at all?
In the past we had some issues with eth-lightwallet as it is no really maintained anymore. This includes:

Not updated dependencies which lead to breaking the library
Pretty big (~ 700kb compressed)
Long waiting time until PRs are merged (we are actually using a fork right now)
Smaller (but a lot of) inconveniences while working with the library
A lot of open issues and PRs
No support of types
Not a lot of tests
Rudimentary documentation

Why ethers.js?

Import and export BIP 39 mnemonic phrases (12 word backup phrases) and HD Wallets (English, Italian, Japanese, Korean, Simplified Chinese, Traditional Chinese; more coming soon)
Connect to Ethereum nodes over JSON-RPC, INFURA, Etherscan, or MetaMask.
Tiny (~88kb compressed; 284kb uncompressed)
Complete functionality for all your Ethereum needs
Extensive documentation
Large collection of test cases which are maintained and added to
Fully TypeScript ready, with definition files and full TypeScript source

Outcome of discussion with @berndbohmeier about this topic:

Migration to ethers.js without breaking current API of clientlib (if possible)
Implementation of custom RelayProvider which extends ethers.providers.Provider and only focus on really needed methods (aka currently supported relay endpoints)
Implementation of custom RelaySigner which extends ethers.signers.Wallet
Remove eth-lightwallet and web3 dependencies

Closed via PR #191


Mesh loading messed up, maybe the indices

Hi.
Any idea what could be wrong? (see picture)

Vertices look right in the place, but the triangles are rendered wrong way.
This is bgfx ported to Marmalade, GLES, , single threaded, on Windows. I did some slight changes to the sources, especially the #defines. But double checking my changes there shouldn't be anything related to this issue.
Spending about 10hours already I have no idea where to move on... Loading looks correct, the command buffers and rendering as well. Still something wrong and I think it's something obvious. All the loaded meshes are similar.
Thanks.

Never mind...

What was it? Wrong data?


Doesn't work on non-english operating systems

Running "Find-Package OpenJS.NodeJS -Provider WinGet" on a german OS (Windows 10, x64, 21H2) does not find any packages. (winget-cli is installed)
exception:
Find-Package : No match was found for the specified search criteria and package name 'OpenJS.NodeJS'. Try
Get-PackageSource to see all available registered package sources.
In Zeile:1 Zeichen:1

Find-Package OpenJS.NodeJS -Provider WinGet

  + CategoryInfo          : ObjectNotFound: (Microsoft.Power...ets.FindPackage:FindPackage) [Find-Package], Exceptio
 n
  + FullyQualifiedErrorId : NoMatchFoundForCriteria,Microsoft.PowerShell.PackageManagement.Cmdlets.FindPackage

What does the WinGet output look like when the following is ran with German localization?
winget search --id=OpenJS.NodeJS.
Is the issue with the column names?

Yes, column names are localized:

In german, the word for "Name and Version" is the same (compared to english), but in other languages that could be different.

Unfortunatley, I dont have any great options at the moment. To your point back on the winget-cli project thread, there aren't any good options I can think of to address this, due to how...fluid WinGet's output is.
Which column names are displayed change depending on the parameters passed, can vary in depth into the input depending on cache state, and are the only thing I've found to semi-reliably parse out the data.
I'm open to a PR to help make it better, but it's not something I'm interesting in boiling the ocean to fix when the underlying CLI utility is so in flux and in such a poor state at the moment.

OK fine I might have fixed it :)
@endpointmanager can you try upgrading the Cobalt module to v0.0.8 to see if that fixes it? I don't have an easy way to test different locales.

I was not able to get it running at my first attempt (tried with Cobalt v0.0.9), but I figured out why ...

Please replace "Get-WinSystemLocale" with "Get-UICulture" ...
$locale = (Get-UICulture).Name
...because "Get-WinSystemLocale" returns the locale (e.g. used for DateTime-Format and so on), and does not represent the display language (e.g. used in winget output)
(In my case, I'm using "en-US" as culture, and "de-DE" as display language)

Not sure, if thats an issue with Crescendo, but the output of the process did not matched some of the Header-Names of the xml, because of the wrong encoding of the winget output.

E.g. "AvailableHeader"...
from xml = "Verfügbar" (correct)
from winget-output (PS)  = "Verf├╝gbar" (wrong encoding)
from winget-output (ISE) = "VerfÃ¼gbar" (wrong encoding)
Only header names with an umlaut was affected.
Is solved it temporarily by using
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
before I use any of the cmdlets.
However, this works only in a Powershell window, not in ISE.

Thanks, this has been very helpful.
I've updated Cobalt in v0.0.10 to use Get-UICulture and Winget in v0.0.4 to force interpreting native output encoding as UTF8.
Unfortunately there doesn't seem to be a way with GitHub Actions VMs to change the language pack used, so please give it another test when you're able.

Hi Ethan,
Good news...
Looks like you solved the localization-issue :-)
Starting with v0.0.10 (Cobalt-Module) and v0.0.4 (WinGet-Module), I can use the WinGet Package Management Provider on my german-localized client, and I think it should work with other languages too.
Greetings,
Paul

Awesome news, thanks @endpointmanager for all your help on this!


Testnet deployment should create a set of private full-nodes to serve infrastructure

Expected Behavior
Blockscout and the Oracle should be run from a set of private full-nodes sequestered behind a firewall in private network.

Related with https://github.com/celo-org/celo-monorepo/pull/2650


Sidekick-documentation

Description
Please include a summary of the change and which issue is fixed. Please also include relevant
motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as
expected)
[x] Documentation (update or new)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can
reproduce. Please also list any relevant details for your test configuration
Testing Checklist

[x] Tested in latest Chrome
[x] Tested in latest Safari
[ ] Tested in latest Firefox

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have requested a review from ... on the Pull Request

Looks good @Kachi-Okorie, @abkenar Please approve.

This is the review document for Ontrack Github to determine which folders fall under what categories


cannot find module 'angular2-json-schema-form'

These are the steps to reproduce the issue:

ng new my-forms
cd my-forms
npm install angular2-json-schema-form --save
add line to my code:

import { JsonSchemaFormModule } from 'angular2-json-schema-form';

The IDE tells me that cannot find de module.  I had a problem with the build process yesterday, and the NPM versions of 0.5.0-alpha.5 and 0.5.0-alpha.6 are completely broken. Thank you for reporting this so promptly.
The immediate problem has been fixed as of NPM version 0.5.0-alpha.7.
In addition, as #66 points out, angular cli builds will still fail if required peer dependencies for Angular Material are not installed too. So I've added these to peerDependencies and updated the readme to reflect this in NPM version 0.5.0-alpha.8 .
Here are the new angular cli install steps, which I just tested as working:
ng new my-forms
cd my-forms
npm install angular2-json-schema-form --save
npm install @angular/cdk --save
npm install @angular/flex-layout --save
npm install @angular/material --save

Then add these lines to your code:
import { FlexLayoutModule } from '@angular/flex-layout';
import { MaterialModule, MdDatepickerModule } from '@angular/material';
import { JsonSchemaFormModule } from 'angular2-json-schema-form';

Now run npm run build (or ng build), and it will work correctly.


[ISSUE #2935] Failed to address the resource file from within Jar as a File

Fixes #2935
Motivation
Fixes the file loading within the jar file
Modifications
Used InputStream to read resource file as a stream
Documentation

Does this pull request introduce a new feature? (yes / no) no

please fix checkstyle error.


Write code to access FOLIO PostgresQL database

This is obviously related to #4, since both are to do with accessing PostgreSQL. But we need to talk to FOLIO first, so we can consult the configuration to even determine the location of the LDP or MetaDB database.

Well, this is turning out to be much more infuriating than I had imagined.
First, we don't have any FOLIO database that is accessible from outside of our AWS environment. So I can't connect to an existing one and use that.
Second, we still don't have a platform-minimal Vagrant box, so I can't just run one locally and reverse-SSH-tunnel into it to get access to the FOLIO database. (I don't have enough memory to run the full-service Everything Box.)
Third, I need to write the PostgreSQL code in mod-reporting to follow the conventions for how we use the database ... but they don't seem to be written down anywhere. They are, I suppose, silently implied by RMB, by the Spring Way libraries, and by folio-vertx-lib ... no doubt, in three subtle divergent ways.
Fourth, @wafschneider tells me that "mod-ldp is evil WRT following FOLIO database conventions. It uses the “public” schema. It does nothing to enforce tenant data separation." So I will probably want to make mod-reporting do something different — which means that it won't be possible to silently cut across from mod-ldp to mod-reporting after all.

I see three possible ways forward, two of them with variants:

I run my own PostgreSQL:

1A. I learn PostgreSQL, learn how to run it locally, learn how to adminstrate it, set up my own development instance, and work against that.
1B. I somewhat short-circuit that by using https://testcontainers.com/ (thanks to @kurtnordstrom for the recommendation)

I run an instance of FOLIO and use its database:

2A. I do the work to finally get platform-minimal over the line, and use that.
2B. I find a way to run platform-complete — most likely by having it run on some remote machine and reverse-SSH-tunnelling into it.

I forego using a FOLIO database at all. Wait, where are you going? Come back! Hear me out!

Number 3 is the most intriguing option. Since I am already committed to making mod-reporting store its data differently from mod-ldp, I can't simply slip in under the covers and read its configuration in the old style. So instead of implementing the /ldp/config WSAPIs on top of the PostgreSQL database, I could just have it delegate to mod-settings.

Let the record show that @wafschneider (in a discussion on Slack) endorses option 3.

All right, this is the plan:

When mod-reporting is enabled for a tenant, it fetches a setting from mod-settings to tell it where the tenant's LDP/MetaDB database is.
It connects that LDP/MetaDB database (so I will still need PostgreSQL code, but that is issue #4, and does not concern us here)
All WSAPI calls to /ldp/config will be proxied through to mod-settings

The only problem with this is that the new code will not be able to read settings stored by the old code. So we will need to provide a script to be used as follows:
$ reporting-config --get-ldp > config.json
[disable mod-ldp; enable mod-reporting]
$ reporting-config --set-reporting < config.json

@wafschneider again, on Slack:

IMO as long as it is clearly documented in the release notes that "this is what you need to do in order to move from mod-ldp to mod-reporting" along with an easy procedure and cross-platform (best effort) code, it should be good. A shell script would be better than Python or Perl.

I will write a shell-script to do this: see #19.

With that requirement estalished, I can close the present issue as WONTFIX. Happily, there is no need for mod-reporting to use the FOLIO database at all!


Filter issue In _createColumnsFilterStatement() instead of or_where() it should be where()

private function _createColumnsFilterStatement($like) {
        $str = "";
        # local filter
        if (!empty($like['local'])) {
            $str = "(" . implode(" OR ", $like['local']) . ")";
        }
        if (!empty($like['global'])) {
            $str = $str . ($str != "" ? ' AND ' : '') . "(" . implode(" OR ", $like['global']) . ")";
        }

        if ($str != "") {
            //$this->ci->db->or_where("(" . $str . ")"); replace with where
            $this->ci->db->where("(" . $str . ")");
        }
        return TRUE;
    }

Filter  must be applied for both clause 'defined by user and search from application '
if we use

or_where :  return rows without  filtering user defined clause
where : return rows if both where clause's are true.

I'm not sure where that code is in DataTables. I don't immediately recognise it. It looks like it might be some CodeIgniter code? Is that correct? If so, this isn't the correct place to post issues since I haven't published any CI code.

No feedback so going to close this issue.


why the code of statemachine generate if-else statement instead of case statement with Verilog

there is no case in the generated Verilog statement, I usually use the case when writing the state machine with Verilog. So, chisel3 still can not generate Verilog code with case？

That's correct, Verilog code generated from when/.elsewhen/.otherwise or switch/is will map to Verilog if/else blocks. Fortunately, from the perspective of synthesis tools, this shouldn't matter in terms of QOR: the tooling should optimize these accordingly.
However, this is clearly sub-optimal in terms of readability of generated Verilog.
The addition of chisel3.experimental.ChiselEnum provides both (1) better support for enumerated types and (2) annotates enumerated Chisel types. With both of these supported, there's been a long-standing idea to write an FSM library that would generate more readable state-machines in the output Verilog.
Summarily, this is a cool idea that's looking for a developer to drive it.

+1 on this. I stumbled over a performance issue that the logic of the switch statement can fit into one LUT per bit in an FPGA, but the prioritized selection chain resulted in 2 LUTs per bit. This doubles the hardware size and reduced the fmax compared to a pure Verilog/VHDL implementation of the same circuit.

This seems to be decently answered. Please re-open if more information is needed here.

I think discussion around this has moved to: https://github.com/freechipsproject/chisel3/issues/1198.
FIRRTL has merged support to generate else if generation via https://github.com/freechipsproject/firrtl/pull/1204. This is available in FIRRTL 1.2.1 (which is used by Chisel 3.2.1).


Reuse more classes between non-forked runs

steps
default setup, run a main class or tests
problem
on every run a new classloader is created with all of classpath (except for scala-library according to this https://github.com/sbt/sbt/issues/4039 )
this wastes metaspace and discards jit results between runs
expectation
as much as possible is reused, perhaps only build products are loaded in new classloader on each run, and the rest of fullClasspath is retained in a hot reusable classloader
notes
sbt version: 1.1.4

To potential contributor: sbt emulates run as a function call on a separate classloader from the build. Much of the necessary information should be located inside https://github.com/sbt/sbt/blob/1.x/run/src/main/scala/sbt/Run.scala.
There's a other part that does the whole thing in the background (aka bgRun), which is in https://github.com/sbt/sbt/blob/1.x/main/src/main/scala/sbt/internal/DefaultBackgroundJobService.scala, but likely you won't need to mess with that part.
@dwijnand Do you think could be a candidate for "good first'?

I think not, because of the performance implications and the lack of out-the-box automated performance analysis.

Fixed in sbt 1.3.0.


delete interest, MultipleRowsFound exception (latent?)

Message type:       ERROR
Location:           /usr/local/lib/python3.9/site-packages/loutilities/tables.py:754
Module:             tables
Function:           wrapped_f
Time:               2023-05-20 11:15:35,914
User:               ********

Message:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/loutilities/tables.py", line 740, in wrapped_f
    self.editor_method_postcommit(request.form)
  File "/app/webmodules/views/userrole/userrole.py", line 40, in editor_method_postcommit
    update_local_tables()
  File "/app/webmodules/model.py", line 75, in update_local_tables
    localtables.update()
  File "/usr/local/lib/python3.9/site-packages/loutilities/user/model.py", line 292, in update
    self._updateuser_byinterest()
  File "/usr/local/lib/python3.9/site-packages/loutilities/user/model.py", line 228, in _updateuser_byinterest
    localuser = self.localusermodel.query.filter_by(user_id=user_id, interest_id=interest_id).one()
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2870, in one
    return self._iter().one()
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 1522, in one
    return self._only_one_row(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 614, in _only_one_row
    raise exc.MultipleResultsFound(
sqlalchemy.exc.MultipleResultsFound: Multiple rows were found when exactly one was required

this was due to database entry created with interest_id=null, when earlier version of software didn't support interests. When retrieve of interest_id=None was requested, all rows which matched user_id were retrieved, not one.

but when the interest is actually deleted, this leaves null in the interest_id, so the same behavior will be seen.


Existing branches are not shown in 'Create branch' dialog

Steps are:

create new branch named 'test'
Before 'Local branches' and 'Remote branches' get refreshed, click 'Create' button'.
Then, the dropdown list of 'Branch' does not show any existing branches.

done.


There are a lot of tensorflow warnings when starting rasa

A general insight from the Rasa 2.0 user tests
Tobias wrote in the user feedback doc:

high prio in my opinion (might be already done 🤔)

Thanks for raising this issue, @tttthomasssss will get back to you about it soon✨
Please also check out the docs and the forum in case your issue was raised there too 🤗

@erohmensing Can you please add @abhilasharoy to the Rasa team so sara-tagger remains silent?

Doesn't seem to be a case any longer (tried it with 2.0.0rc3). Could you confirm it with 2.0.0rc3, @abhilasharoy or @wochinge?

@dakshvar22 did you already fix this?

Do we have the exact tensorflow warnings that came up in the user tests? if they were related to tensorflow-addons being not supported completely by the tensorflow version, then that should already be fixed when we updated the version of tensorflow-addons.

Checking with rc3

I can confirm the warnings are fixed 👍


Tiny testing setup for the Relay compiler

This just adds a very simple Runt test for the Relay compiler itself (just compiling Relay text-format programs to FuTIL. It doesn't do the FuTIL compilation or Verilog simulation---the idea is that we can start here for now and eventually move the end-to-end tests into the main test suite where all that magic happens.
There is one annoying installation step to use the Relay text format parser---see my note in the README herein.
As a quick Runt mini-tutorial for @ViviYe, the tests directory contains *.relay files, and all Runt does is check that compiling them matches the results in the corresponding *.expect. Here are the important commands:

runt: Run the tests to report failures. (This is a healthy thing to do after making changes to confirm that you didn't break anything. :smiley:)
runt --diff: Compare the current (actual) result to the saved (expected) result. (Use this to find out why something failed.)
runt --save: Take the actual outputs and save them as the expected outputs. (Do this after looking at runt --diff and deciding that those differences are what you wanted to "lock in" your changes.)

A good workflow for adding new tests is:

Add a new *.relay file to the tests directory.
Use runt --diff to look at the output and decide whether it looks OK.
 Particularly useful ones are the --include and --exclude flags which let you filter out test.
For example, you can type:
runt -i  relay-futil/tests/identity.relay

to only run the tests that use identity.relay as an input. This is super useful when you want to focus on a specific test. The --include flag composes with --diff and --save. So you can type the following to just run the identity.relay tests and see their diffs:
runt -i  relay-futil/tests/identity.relay -d


feat(l2g): add coloc based features

This PR includes changes in the L2G pipeline to bring eCaviar based features.

Business logic to extract the feature is practically unchanged, with the exception that the maximum CLPPs in the neighborhood are in logarithmic scale. L2G should learn to prioritise genes where this value is more negative.
Bug fixes in how features are built into the feature matrix
Removal of the overlap dependency. This adds extra logic to extract overlaps only for the study loci in the curation set that doesn't look very pretty.

Codecov Report
Attention: 93 lines in your changes are missing coverage. Please review.

Comparison is base (42b366c) 85.67% compared to head (4265d8b) 85.81%.
Report is 50 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #400      +/-   ##
==========================================
+ Coverage   85.67%   85.81%   +0.14%
==========================================
  Files          89       96       +7
  Lines        2101     2595     +494
==========================================
+ Hits         1800     2227     +427
- Misses        301      368      +67

Files
Coverage Δ

src/airflow/dags/common_airflow.py
90.38% <100.00%> (ø)

src/airflow/dags/finngen_preprocess.py
100.00% <100.00%> (ø)

src/airflow/dags/gwas_catalog_harmonisation.py
43.47% <ø> (ø)

src/airflow/dags/gwas_curation_update.py
100.00% <100.00%> (ø)

src/otg/common/session.py
87.50% <100.00%> (+0.32%)
:arrow_up:

src/otg/dataset/dataset.py
91.80% <100.00%> (ø)

src/otg/dataset/l2g_feature_matrix.py
82.92% <100.00%> (+7.31%)
:arrow_up:

src/otg/dataset/l2g_prediction.py
90.90% <100.00%> (+0.43%)
:arrow_up:

src/otg/dataset/study_locus.py
96.20% <100.00%> (+0.04%)
:arrow_up:

src/otg/datasource/finngen/study_index.py
100.00% <100.00%> (ø)

... and 25 more


DryIoc 4.1.0-preview1 singleton decorators are not getting called

DryIoc 4.1.0-preview1 singleton decorators
Singleton decorators are ignored for compiled expressions...

I will try to write unit tests for this...

This is unfortunate, likely due reordering in GetExpressionOrDefault or allowing to compile singleton decorators into Constant.
Yes, we need the test, cause currently everything is passing.

@dadhi I opened PR with test case for this behavior.

@dadhi Did you have a look at this issue yet? Is it difficult one?

I have fixed it. will push today.

I have published v4.1.0-preview-02 with the fix and the FEC with your PR.


Add support for GetAttributeAtVertex for D3D

This operation was added along with the SV_Barycentrics system-value input, and allows for a nointerpolation varying input to a fragment shader to be fetched at a specific vertex index within the primitive that is causing the fragment shader to be invoked.
This change adds support for the new operations in the standard library, and also includes a test case to make sure that we emit it correctly when producing HLSL/DXIL.
This change also includes a small bug fix to our emission logic for function parameters so that we properly emit layout-related attributes for varying parameters declared directly on an entry point.
(Note that most attribute end up being declared in struct types in existing HLSL shaders, and our IR passes produce only global variables for attributes on GLSL; the only case this affects is inidividual scalar/vector attributes declared declared as entry-point parameters, when outputting HLSL)
Note that this change only adds support for the new function on the HLSL/DXIL path, and doesn't yet add any cross-compilation support for GLSL/SPIR-V. The reason for this is that the equivalent GLSL feature(s) appear to use a different model to the HLSL version, and we need to invent a suitable approach to align them to make portable code possible.

@csyonghe Yes, we eventually need some kind of semantic checking for the validity of calls to GetAttributeAtVertex(), EvaluateAttrributeAtCentroid(), etc.
The simplest validation rule would be entirely syntactical: a valid argument must be a "attribute expression" where:

Varying inputs to a fragment shader entry point are attribute expressions
Member access base.field on an attribute expression yields an attribute expression'
Array indexing on an attribute expression yields an attribute expression

That syntactic formulation might be a bit limiting since it doesn't allow for these operations anywhere except the top level of a fragment shader entry point, but they'd be easy-ish to enforce (we could add an attribute on the appropriate function parameter and then perform these checks around the same time we do the checks for out and inout arguments).
(The big thing that the above discussion leaves out is that some of these evaluation operations may place restrictions on what interpolation qulaifier(s) are allowed on the attribute, implying that our type system should also be propagating and tracking those qualifiers - but only for accesses that start with a fragment-shader varying input)
A more complex but complete solution would introduce one or more type qualifiers that indicate attribute-ness of the appropriate kind (e.g., flat interpolation vs. perspective-corrected linear interpolation, etc.) and then have type-system features for propagating those qualifiers through the relevant operations. Basically an attribute T should be able to be coerced to a plain T, but not vice versa, and then specific ops (like field access and array indexing) would know to propagate such attributes from the base type through to the result. We'd probably need to make overload resolution aware of such conversions, so that they can prefer an exact qualifier match over a match on type but not qualifier.
The thing that keeps getting at me is that such attributes are basically just rate qualifiers, so if we are going to introduce a notion of a QualifiedType we might as well call it something like a RateQualifiedType and have the rates themselves come rom syntactic declarations that can have their own associated rules...


Usiing Django rest framework

im working with tinymce on my application which a description is saved using it . but the issue I'm facing is the endpoint returns the content with HTML format in it , which is not cool for me , I want to return only the description content with out the HTML content.

Sorry, but this looks like more of a support issue, I don't see anything we can do with that report.


MVP2/First Release meta-issue

This is an issue of all of the things we need to do to get to a first release. Feel free to file your own issues and add them to this ticket.

[x] Basic authentication (either better documentation of how to set up HTTP basic auth or our own small in-house solution. The former would be better as it would maintain the lack of database for this application. If anyone knows any solutions for authentication which don't require a database, let me know!). This is issue #26.
[ ] Better naming solution for sites. This means that individual sites, by default, should have a name that matches their site domain name (or a new option). If there is an API call for the thing which @wardi was talking about (the settings panel) we could very well just use ckanapi or a web request to do this. This is issue #25
[x] Update README.md file to reflect recent advancements in the run.sh automated install. Also document manual installation if people don't just want to run a script. Issue #5.
[ ] https://github.com/datacats/datacats/issues/298

So as of c75d7763b249c3a12c07a93d6efdd953ebf36354 we have an authentication scheme which is just a simple password (we can add a username later if needed). It currently uses a session cookie encrypted with the app secret and SHA256 hashing with salt.
I really would like some review on this as my crypto-fu isn't great. @wardi

FWIW, it's really trivial to provide HTTP Basic authentication. You just have to send a couple of HTTP headers to the client, and compare the response to wherever you store the username and password in. I do this every time I think "I need admin authentication for this system I built, but I don't want want it to be a whole thing."

If you'd prefer HTTP basic auth, I can do that, it's just that I'm not familiar with it.

It's all the same to me. :)

So we just had a little bit of a discussion in the office regarding actually naming the sites.
There is now a 'display name' field in the ckan-multisite admin panel, and it is pretty easy to change. Problem is, it doesn't actually impact the real site name. We have a couple options here:

Wait for the better option of using the database-based settings in CKAN 2.4.
Use a simple insertion into the database in the same place that CKAN stores the site title from the preference panel that Ian was talking about. Hacky, but this is probably the best solution for automated deployments on CKAN 2.3.
Give a template title for it and let the user set it to what they'd like (easiest, but least automated)

Thoughts, @waldoj?

Hmm. There are no great options here, but ¯_(ツ)_/¯. Point by point:

What's the timeline for CKAN 2.4?
I agree entirely with your assessment.
I don't follow. Could you explain the UX here? It sounds like this might be egregious.

CKAN 2.4 itself is out but we are waiting for our release manager to come back to put out a release of the datacats tool

Re: 3. is similar to what you suggested before, call any new site "A Multisite Site" and in the "promoted" section we mention that they should change the site title in there

CKAN 2.4 itself is out but we are waiting for our release manager to come back to put out a release of the datacats tool

Got it. I'm much less confused now. :)

is similar to what you suggested before, call any new site "A Multisite Site" and in the "promoted" section we mention that they should change the site title in there

Ahh, I see—so it remains possible for people to change the name on a new site, this is just about eliminating that step.

So, this is my preference, but ultimately I'm happy to defer to your judgment: Option #2, with the intent of ripping it out when CKAN 2.4 support obviates that hack. It's ugly, but I suspect it'd be quick and easy, and likewise easy to remove down the line, if it's anything like the "I'll just directly write this thing to the database" hacks I've been guilty of over the years. :)

Yup, I was trying it out today. The only catch is that we may need to shut down CKAN itself to change the site name as it will not reread from the database automatically - this is not ideal.

@JackMc you make us sounds so formal, heh. Ian is gone camping for a few days and will be back Tuesday to push out a new release. That's the alternate timeline

So perhaps the site name is set at the time that it's created, and future changes must be performed through the deployed CKAN site's admin section? That way, the only restart required is in the process of establishing the site in the first place, delaying its launch briefly. I think that's a perfectly reasonable solution for an MVP.

Sounds good!

@deniszgonjanin Sshh, next you're gonna let it slip that I don't wear a suit to work!

Ian is gone camping for a few days and will be back Tuesday to push out a new release.

Yay for camping. :) If the alternative is just waiting a few days until Ian can get out a new datacats release, that sounds great.
The wonderful thing about working in open data is there's no such thing as an open data emergency. :) Basically everything can wait.

There isn't much needed here technical-wise. If you could have another go at this @waldoj that'd be lovely! :)

I'm gonna close this issue.


Python logging should use deferred formatting

Right now, there are lots of log statements that use f-strings or % expressions that ~expensively evaluate, just to be discarded by the logger because of the log level. At one point someone benchmarked that and figured out that we'd get substantial gains from wrapping a bunch of those in calls to logger.isEnabledFor(level).  Well, that's not necessary; the logger can do that for us, if we let it defer the formatting to the logger.
@haq204 wrote:

just to add even more the needless complexity to python logging, f-strings squash things to a single string that gets added to the Log record so usually prefer to use old-school %s because the logging library is optimized to defer formatting until it needs to print it out with that format though you can change the Formatter on your logger instance.
It's also relevant when you use error log aggregators like Sentry for cleaner error log groupings if they are smart enough to deal with the actual log objects and not the message string
Relevant references:

https://docs.python.org/3/howto/logging-cookbook.html#formatting-styles
https://docs.python.org/3/howto/logging.html#optimization

TLDR: python logging is complicated

@LukeShu wrote:

@haq204 So the lesson isn't "go back to using old-school log("fmt" % args)", it's "use old-school-style 'fmt' with log("fmt", args)". Good to know.
Is there a linter that can check for formatted strings (f-strings, or % expressions) being passed to the logger, instead of passing the args as args?
And, FWIW, I wouldn't say that's complicated, it's a totally expected optimization (we do it in dlog!), just one that apparently none of us realized Python makes available to us.

It looks like this is what PyLint W1202 is checking for.  We should start running PyLint.

At one point someone benchmarked that...

That would be me. 🙂 And the self.debug or self._log_debug shortcut was about typing fewer characters, and about not constantly making the function call for logger.isEnabledFor(level).
If you look at the history, you'll see more than a few places where I later undid that wrapper in favor of the old-school "%s", arg-style logging... but yeah, it was never worth finding them all, and there are still places where the wrapper is better just because there's so much logging we want to do in debug mode.
All that said, though: yeah. We should run PyLint.


Move from travis to github actions

I believe travis is defunct.  Let's move the tests in .travis.yml into a github action (also replace badge in README.md)

@yarikoptic There's already a GitHub Actions test workflow in this repository.  What more do you need for this?

right -- " Additional tests" - to test README.md.
Keep that one but also make "Tests" with following from travis setup:
install:
  - travis_retry ./travis/install.sh

script:
    - grep -m 1 -l '^#!/bin/.*sh' | grep -v '\.\(txt\|md\)' | xargs shellcheck
    - bats -t scripts/tests

steps

@yarikoptic So you want a separate workflow with a title of "Tests"?  Normally I'd name it test.yml or tests.yml, but there's already a tests.yaml here, so what should the filename be?

I don't care much. base-tests.yml?


 For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
bin /home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so
bin /home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so
/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /home/amd00/anaconda3/envs/gh_trl did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/tmp/torchelastic_fmnkd2px/none_cih6y_hj/attempt_0/1/error.json')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get CUDA error: invalid device function errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary /home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so...
/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /home/amd00/anaconda3/envs/gh_trl did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/tmp/torchelastic_fmnkd2px/none_cih6y_hj/attempt_0/0/error.json')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get CUDA error: invalid device function errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary /home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so...
Found cached dataset parquet (/home/amd00/.cache/huggingface/datasets/lvwerra___parquet/lvwerra--stack-exchange-paired-ea956f7e49277b88/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)
Found cached dataset parquet (/home/amd00/.cache/huggingface/datasets/lvwerra___parquet/lvwerra--stack-exchange-paired-ea956f7e49277b88/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)
Found cached dataset parquet (/home/amd00/.cache/huggingface/datasets/lvwerra___parquet/lvwerra--stack-exchange-paired-6fbcbcc16115b7c8/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)
output_name : _peft_stack-exchange-paired_rmts__100000_2e-05
Found cached dataset parquet (/home/amd00/.cache/huggingface/datasets/lvwerra___parquet/lvwerra--stack-exchange-paired-6fbcbcc16115b7c8/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)
output_name : _peft_stack-exchange-paired_rmts__100000_2e-05
script_args.model_name: /home/amd00/llm_dev/trl/llama-se/final_checkpoint/
Traceback (most recent call last):
File "/home/amd00/llm_dev/trl/examples/stack_llama/scripts/reward_modeling.py", line 139, in
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name, use_auth_token=True)
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 711, in from_pretrained
return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1796, in from_pretrained
raise EnvironmentError(
OSError: Can't load tokenizer for '/home/amd00/llm_dev/trl/llama-se/final_checkpoint/'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/home/amd00/llm_dev/trl/llama-se/final_checkpoint/' is the correct path to a directory containing all relevant files for a LlamaTokenizerFast tokenizer.
script_args.model_name: /home/amd00/llm_dev/trl/llama-se/final_checkpoint/
Traceback (most recent call last):
File "/home/amd00/llm_dev/trl/examples/stack_llama/scripts/reward_modeling.py", line 139, in
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name, use_auth_token=True)
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 711, in from_pretrained
return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1796, in from_pretrained
raise EnvironmentError(
OSError: Can't load tokenizer for '/home/amd00/llm_dev/trl/llama-se/final_checkpoint/'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/home/amd00/llm_dev/trl/llama-se/final_checkpoint/' is the correct path to a directory containing all relevant files for a LlamaTokenizerFast tokenizer.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 3184) of binary: /home/amd00/anaconda3/envs/gh_trl/bin/python
Traceback (most recent call last):
File "/home/amd00/anaconda3/envs/gh_trl/bin/torchrun", line 8, in
sys.exit(main())
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/torch/distributed/run.py", line 762, in main
run(args)
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/torch/distributed/run.py", line 753, in run
elastic_launch(
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/amd00/anaconda3/envs/gh_trl/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
examples/stack_llama/scripts/reward_modeling.py FAILED
Failures:
[1]:
time      : 2023-07-09_11:33:08
host      : MZ32-00
rank      : 1 (local_rank: 1)
exitcode  : 1 (pid: 3185)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Root Cause (first observed failure):
[0]:
time      : 2023-07-09_11:33:08
host      : MZ32-00
rank      : 0 (local_rank: 0)
exitcode  : 1 (pid: 3184)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Hi @SeekPoint
Thanks for the issue? What is the transformers version you are using?
Altertanitvely you can manually save the tokenizer on the corresponding folder and run the script. Also make sure to properly format the error trace by wrapping it inside code blocks "" in the beginning and "" in the end.

(gh_trl) amd00@MZ32-00:~/llm_dev/trl$ pip list | grep trans
transformers                  4.29.2

failed at transformers 4.30.2 yet

why del the example of stack_llama?

We didn't delete - we just moved it: https://github.com/lvwerra/trl/tree/main/examples/research_projects/stack_llama/scripts


Install Kube-proxy service with calico

Replicate these steps on PS1 files and smoke test it

/assign

Added on https://github.com/kubernetes-sigs/sig-windows-dev-tools/pull/84
/close


Implement dropDatabase command.

Closes #111.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Good start. I'd be interested in what @AlekSi thinks in terms of testing this, as we don't currently have a create database command?

Thx, for review, I'll fix this errors

One more thing to note: what about test? Something like this:
* https://github.com/FerretDB/FerretDB/pull/62/files#diff-efe5633155496149a2523f825232224f1dfc096d50a197246fe972d464f0e262R33

Yep, tests would be nice. We should test with both existing and non-existing databases.
(Databases are created on the fly, so to create a database, we should insert some dummy document in some dummy collection)

fixed

There is a merge conflict now. May you please resolve it?

There is no need to do a force-push – the PR will be squashed on merge anyway, and force-pushes require me to re-approve CI checks every time.

sorry, next time  will be more tidy

Please tell when it is ready for review :)

Please tell when it is ready for review :)

Ready for review)

Thank you!

Thank you!

There is some golangci-lint problem. Do i need to fix them?

Don't worry, I will fix it


Move the detailed feature planned implementation breakdown section to its own file in /docs, link to it in the readme, and just have a simple list for people to read.

It's getting really long now, and I didn't think it would end up like this when originally adding to each item.

This is done as of last night or something.


Fixed footer at the bottom

Is there a chance, that there will be something similar to nav. bar, but for the footer? And it would be great to have a way to easily stick it at the bottom.

@rodrigomuniz But then if I resize browser window, footer flows over other content, instead I would like to hide itself when it meats some content on the page. Link in sticky footer example.

Solution here __https://github.com/twitter/bootstrap/issues/306#issuecomment-4272257__

This is something that is easy enough to do, but would be a really nice feature in bootstrap.

hello world


remove dot from extension when using file restrictions

when a file restriction extension is noted including the DOT, remove the DOT because we don't take that into account in our code and things don't work with DOT in.

Make sure we note this also specifically in the documentation. Don't add the dot because we will remove it when you do :).

When file restrictions are set with a dot, the dot is removed from the file value.


Register with Bower package manager

http://bower.io/docs/creating-packages/#register
Other issues that also request this: https://github.com/lorenooliveira/ng-text-truncate/issues/6 https://github.com/lorenooliveira/ng-text-truncate/issues/9 https://github.com/lorenooliveira/ng-text-truncate/issues/18
https://github.com/jeromev/ng-text-truncate/commit/8485c37f30936e4da2a9a768399065373b3d2959

:+1:

+1

+1


Semantic issues with versions > 5.1.0

Goals
Use a version of RealmSwift > 5.1.0
Expected Results
No error building app after installing > 5.1.0
Actual Results
The app fails to compile with a bunch of semantic issues:
...app/Pods/Realm/include/core/__bit_reference:176:62: No member named '__libcpp_ctz' in namespace 'std::__1'; did you mean '__libcpp_wcsstr'?
...app/Pods/Realm/include/core/__bit_reference:1119:7: No template named '_If'
etc

Steps for others to Reproduce
Execute pod update with pod 'RealmSwift' in the Podfile
Try to run the app and it will throw errors
Specify pod 'RealmSwift', '5.1.0' instead and it runs successfully
Version of Realm and Tooling
Realm framework version: 5.3.1 or anything > 5.1.0
Xcode version: 11.6 (11E708)
iOS/OSX version: iOS 13.6, macOS 10.15.6
Dependency manager + version: CocoaPods 1.9.3
Let me know if you need any more info!

Do you have Xcode 12 installed and set as the default Xcode version? pod install needs to be run with the same Xcode version as you're going to build the project as we set things up differently for each of them. Deleting the Pods directory and rerunning pod install with Xcode 11 active selected should fix it.
If you don't have Xcode 12 installed at all then that'd mean there's a bug in how we're detecting the active Xcode version.

I do! Switched to Xcode 11 via xcode-select, made sure command line tools is Xcode 11, deleted Pods and reran pod install but I still get the same errors. 🤔

I am see a similar issue in Xcode 11.5 but only with 5.3.2. I was able to revert to 5.3.1 and the issue went away. 5.3.2 builds correctly in Xcode 12 beta 3.

This should go away in the next release as the bug we were working around (incorrectly?) appears to have been fixed in Xcode 12 beta 3.

This has been fixed. I tested with 5.3.5 and Xcode 12 beta 5.