Other plugins failing to load

EDIT: solved (partially)
It turns out that there was a failure to decrypt some of the .obsidian files. I will leave more details in a comment.
Abstract
Some (but not all) plugins are failing to load on start. The plugin folders appear intact in the filesystem, but not within the app.
I'm not 100% certain that LiveSync did cause this issue; It's very possible that this was totally my own error or another plugin caused it. However, it happened at the same time that I was messing with various LiveSync settings, and I hadn't made any real changes to other plugins or Obsidian in the several hours before the issue happened.
Expected behavior

Plugins visible in the Community Plugins interface match the plugins in .obsidian/plugins
Installed plugins don't disappear without a reason

Actually happened

Over half of the community plugins are not visible in the Obsidian Settings sidebar or the Community Plugins settings pane
Plugin functionality is not loading for these plugins
Some plugins remain functional, unclear why

Reproducing procedure
I have no idea what caused it, but I'll provide environment details below. The issue is persistent.
Report materials
Went over the length limit...
report-data.txt
Screenshots
Server config check:

Other information, insights and intuition.
Obsidian plugins
In GUI
The plugins shown in the Community Plugins GUI in Obsidian:

In JSON
The plugins listed in .obsidian/community-plugins.json:
[
  "github-embeds",
  "obsidian-excalidraw-plugin",
  "obsidian-minimal-settings",
  "obsidian-style-settings",
  "obsidian-kanban",
  "obsidian-tasks-plugin",
  "dataview",
  "execute-code",
  "templater-obsidian",
  "recent-files-obsidian",
  "obsidian-mind-map",
  "email-block-plugin",
  "obsidian-outliner",
  "omnisearch",
  "obsidian-linter",
  "nldates-obsidian",
  "better-word-count",
  "highlightr-plugin",
  "table-editor-obsidian",
  "obsidian-vimrc-support",
  "vim-yank-highlight",
  "obsidian-relative-line-numbers",
  "vim-toggle",
  "colored-tags-wrangler",
  "obsidian-icon-folder",
  "media-extended",
  "obsidian-link-embed",
  "dataview-serializer",
  "obsidian-advanced-uri",
  "meld-encrypt",
  "obsidian-auto-link-title",
  "links",
  "obsidian-local-images",
  "wikipedia-search",
  "obsidian-dangling-links",
  "quick-preview",
  "boost-link-suggestions",
  "obsidian-full-calendar",
  "calendar",
  "obsidian-text-format",
  "obsidian-regex-replace",
  "copy-as-html",
  "find-and-replace-in-selection",
  "remember-cursor-position",
  "obsidian-cursor-location-plugin",
  "advanced-cursors",
  "smart-connections-visualizer",
  "homepage",
  "advanced-canvas",
  "canvas-mindmap",
  "obsidian-raindrop-highlights",
  "janitor",
  "random-numbers-generator",
  "obsidian-dice-roller",
  "improved-random-note",
  "share-note",
  "obsidian-livesync"
]

In local filesystem
The content of .obsidian/plugins/:
.
|-- advanced-canvas
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- advanced-cursors
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- better-word-count
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- boost-link-suggestions
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- calendar
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- canvas-mindmap
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- colored-tags-wrangler
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- convert-url-to-iframe
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- copy-as-html
|   |-- main.js
|   `-- manifest.json
|-- dataview
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- dataview-serializer
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- email-block-plugin
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- execute-code
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- find-and-replace-in-selection
|   |-- main.js
|   `-- manifest.json
|-- github-embeds
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- heading-level-indent
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- highlightr-plugin
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- homepage
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- improved-random-note
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- ink
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- janitor
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- link-exploder
|   |-- main.js
|   `-- manifest.json
|-- link-favicon
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- links
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- media-extended
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- meld-encrypt
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- nldates-obsidian
|   |-- main.js
|   `-- manifest.json
|-- numerals
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-advanced-uri
|   |-- main.js
|   `-- manifest.json
|-- obsidian-auto-link-title
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-cursor-location-plugin
|   |-- main.js
|   `-- manifest.json
|-- obsidian-dangling-links
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-dice-roller
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-excalidraw-plugin
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-full-calendar
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-git
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-icon-folder
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-importer
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-kanban
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-link-embed
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-linter
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-livesync
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-local-images
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-math-plus
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-mind-map
|   |-- main.js
|   `-- manifest.json
|-- obsidian-minimal-settings
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- obsidian-outliner
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-raindrop-highlights
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-regex-replace
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-relative-line-numbers
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-style-settings
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-tasks-plugin
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-text-format
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-vimrc-support
|   |-- main.js
|   `-- manifest.json
|-- omnisearch
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- quick-preview
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- random-numbers-generator
|   |-- main.js
|   `-- manifest.json
|-- recent-files-obsidian
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- remember-cursor-position
|   |-- cursor-positions.json
|   |-- main.js
|   `-- manifest.json
|-- share-note
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- smart-connections
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- smart-connections-visualizer
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- solve
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- system3-relay
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- table-editor-obsidian
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- tag-wrangler
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- templater-obsidian
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- vim-toggle
|   |-- main.js
|   `-- manifest.json
|-- vim-yank-highlight
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
`-- wikipedia-search
    |-- data.json
    |-- main.js
    `-- manifest.json

Potential causes

Multiple rebuilds, merges, and fetches while I was troubleshooting other issues
Quitting Obsidian during data operations
Briefly enabling Customization Sync for LiveSync's own plugin data files
Toggling options before I understood what they did
A couple days ago I accidentally put self-referencing symlinks in the .obsidian folder, and LiveSync hit a recursion error trying to parse them. I removed these and LiveSync seemed to recover fine.
Laptop and phone both running LiveSync simultaneously
General recklessness

Attempts to fix

LiveSync > rebuild local db and remote db from local files
LiveSync > fetch from remote db
Disabling LiveSync and relaunching obsidian
Disabling LiveSync and duplicating the vault locally
Uninstalling LiveSync and relaunching
Uninstalling LiveSync and duplicating the vault locally
Reinstalling Obsidian and opening both vaults
Created a vault with only the .obsidian directory and no other files
Verified file permissions for all files in .obsidian/

While LiveSync seems to have caused the problem initially, the problem seems persistent in the vault contents, probably
Other observations

Multiple syntax errors in the Obsidian console, these might each correspond to an individual failure to load a plugin?
An extraneous [] in syncIgnoreRegEx: \.DS_Store|[]|^raindrop\/ in livesync-config
The size of the remote went from ~600MB to ~150MB around the same time, but I can't say for sure if this is related, since I was implementing ignore patterns which might have removed files separately from this issue.

The end
I do have backups and nothing was lost in this case thankfully!
Thanks so much for your hard work on this plugin; It's certainly the best open source option for Obsidian syncing I've seen, and I'd be interested in contributing once I get it all figured out.
EDIT: Solved partially
See comment below

Update:
Within 30 seconds of finishing writing the initial bug report, I finally decided to actually look inside the vault files in .obsidian, and voila, It looks like around half the files in .obsidian were left encrypted at some point, for example:
./plugins/homepage/manifest.json:
ewoJImlkIjogImhvbWVwYWdlIiwKCSJuYW1lIjogIkhvbWVwYWdlIiwKCSJ2ZXJzaW9uIjogIjQuMC43IiwKCSJtaW5BcHBWZXJz[...]

./plugins/better-word-count/main.js:
J3VzZSBzdHJpY3QnOwoKdmFyIG9ic2lkaWFuID0gcmVxdWlyZSgnb2JzaWRpYW4nKTsKdmFyIHN0YXRlID0gcmVxdWlyZSgnQGNv[...]

./plugins/obsidian-raindrop-highlights/main.js:
LyoKVEhJUyBJUyBBIEdFTkVSQVRFRC9CVU5ETEVEIEZJTEUgQlkgRVNCVUlMRAppZiB5b3Ugd2FudCB0byB2aWV3IHRoZSBzb3Vy

It seems likely that the direct cause had to do with interrupting LiveSync at the wrong time, probably by quitting Obsidian. I don't really know how the decryption process is handled though.
If possible, it would be great to have some safety checks in place to prevent unencrypted data from being overwritten by encrypted data, or for interrupted processes to resume gracefully. In the meantime I'd recommend some simple warning text next to the end-to-end encryption setting.
For now I'll disable encryption and rebuild the vault, and should be smooth sailing from there on.

Thank you for opening this issue! And so sorry for this!
It seems that the files had been decrypted successfully. However, it had not been decoded as a binary files.
(The file content that your have pasted indicate this situation! Thank you for your cooperation!)
I still wonder why it had been occurred, however, once it had been done, it cannot be resolved automatically.
Would you mind if I ask you to uninstall these plug-ins (from all devices) and install again?


undefined reference to SCPITMCTransport

Lubuntu 22.04
Latest scopehal-apps as of today pulled, cmake ran fine after all dependencies installed per getting starting guide, but make results in the following (a couple places).
/usr/bin/ld: ../../../lib/scopehal/libscopehal.so: undefined reference to `SCPITMCTransport::GetTransportName[abi:cxx11]()'
/usr/bin/ld: ../../../lib/scopehal/libscopehal.so: undefined reference to `SCPITMCTransport::SCPITMCTransport(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [src/examples/curvetrace/CMakeFiles/curvetrace.dir/build.make:133: src/examples/curvetrace/curvetrace] Error 1
make[1]: *** [CMakeFiles/Makefile2:1615: src/examples/curvetrace/CMakeFiles/curvetrace.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

This is the same bug as #703, closing as duplicate.


How to change the wheel animation during boot?

Laravel Version: 5.4
Voyager Version: new
PHP Version: 7.1
Database Driver & Version: MySQL 5.7

Description:
How i may change loading image?

The setting for this is admin_loader.

where i can exactly find the admin_loader ?

It should be on the settings page? Isnt it there?

I cant find it either in the voyager folder neither in the app.config file

What about the settings table? See it there?

nothing is there ... the table is empty

Will be back with a solution tomorrow.

okey 👍  Thanks

Okay, you have the following options when it comes to getting this setting.
First you can add the setting manually by going to /admin/settings and create the following setting.

Second you can get all Voyager default settings by running the settings seeder.
php artisan db:seed --class SettingsTableSeeder


Add overdose party by furyan

Song Name
overdose party
Artist Name
furyan
Source
Youtube
Youtube Link
gxJAiZH-5XOsVW08
Direct File Link
No response
Song ID
0
Start Offset [ms]
1
Extra
Created from Auto Nong.
Created for level: 107394885

reject
wrong song id


Add hyper-savetext to Hyper store

This updated pull request has a preview image with 1200x800 dimensions

Thank you so much for your contribution!
I have a few requests, if possible:

If you could rename your package to hyper-savetext. hyperterm is the old name for Hyper and we can't use or promote that name anymore.
If possible, make your preview image 600x400 pixels with 2x density (so it turns out being 1200x800 pixels). If you need help with this, I'm happy to supply an image with your plugin!
Please make it so your image doesn't have a white background. If the background was transparent or had an image as the background (like from unsplash.com) that would be much better. Again I'm happy to help out there.

@timothyis I've renamed the package to hyper-savetext, and uploaded a picture that doesn't have a white background, but I'm not sure how to address your second point. The preview image is 1200x800, but I'm not sure how to make it 600x400 with 2x pixel density (I always thought pixel density was decided by the device displaying the image, not by the image itself).

@neil-orans Could you submit those new changes? Then we'll see 🙏
Also yes, I need to update the wiki on how to create different density screenshots. I'll work on that this week 🙏

Changes pushed. Thanks @timothyis

Thank you!


##[error]TypeError: Cannot read property 'updateAndMonitorAppSettings' of undefined

I have a task in my pipeline as below ..

task: AzureWebApp@1
displayName: 'Azure Web App Deploy: MS-us-web-daps-api-1'
inputs:
azureSubscription: $(azureSubscription)
appType: webAppLinux
appName: $(webAppName)
package: '$(System.ArtifactsDirectory)/drop/**/target/*.?(war|jar)'

I get the error ##[error]TypeError: Cannot read property 'updateAndMonitorAppSettings' of undefined
No other information is provided to understand the error better.
I dont know if this is a bug or a mis configuration or issue with permissions.. Appreciate any feedback on this.
Thanks
Sanjay

can you try giving exact extension of package instead if this ?(war|jar)?

@mssanjay any update on this?

Closing this issue. Do try above solution. Let me know if it doesn't work


[full-ci] Do not share versions

Description
Change: Do not share versions
We changed the default behavior of shares: Share receivers have no access to versions. People in spaces with the "Editor" or "Manager" role can still see versions and work with them.
How Has This Been Tested?

CI

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Technical debt
[ ] Tests only (no source changes)

Checklist:

[x] Code changes
[ ] Unit tests added
[x] Acceptance tests added
[ ] Documentation ticket raised:

:boom: Acceptance test wopiValidatorTests-ocis failed. Further test are cancelled...

@tbsbdr @mmattel This may have product or docs relevancy.

@ScharfViktor Ok, you can take over that PR. Only adapting the acceptance tests is needed.

let me do a rebase.

I should probably fix e2e and bring new webBump to PR

I think I'm done. I bump webCommit and CI should be green. @micbar  could you check if the oc:permissions in the contract tests(changing in the last commit) are correct or not.


Enable GeoJson support in Azure.Search.Documents

Overall looks pretty good, but I'm confused whether this is still experimental or not - which seems it is, since we now reference the Azure.Core.Experimental project. So shouldn't the preproc conditions stay in?

Yes, this Preview release of Search depends on Azure.Core.Experimental.
This is not experimental code however in Search's upcoming preview.

I will revive this PR after fixing the merge issues.


Feature add support to provide custom getToken function

Hello,
we had an issue with current token fetching we need to use our custom fetch function with proxy agent. So I provided way to override current getToken function similar as getUserInfo.

Thanks for feedback! Cool that work for us also, I updated it.

Looks good, thank you!


New Version: Youdao.youdaodict <IP_ADDRESS>

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

I can't find that download link. At https://www.youdao.com/ I see this link http://c.youdao.com/dict/download.html?app=dict&vendor=webdict_default&platform=win&url=http%3A%2F%2Fcodown.youdao.com%2Fcidian%2FYoudaoDict_webdict_default.exe, where did you find the one in the manifest (http://download.ydstatic.cn/cidian/static/8.10/20210915/YoudaoDictSetup.exe) ?

I can't find that download link. At https://www.youdao.com/ I see this link http://c.youdao.com/dict/download.html?app=dict&vendor=webdict_default&platform=win&url=http%3A%2F%2Fcodown.youdao.com%2Fcidian%2FYoudaoDict_webdict_default.exe, where did you find the one in the manifest (http://download.ydstatic.cn/cidian/static/8.10/20210915/YoudaoDictSetup.exe) ?

Go to http://cidian.youdao.com and click download, then copy the link from downloading file. You can see the link has a patternhttps://download.ydstatic.cn/cidian/static/<first two parts of version number>/<update time>/YoudaoDictSetup.exe

I see that now, thanks!

/azp run


Feature: Table redesign

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Changed hover effect, added search bar in 2 pages and added button as per requirement.

Why was this change needed? (You can also link to an open issue here)
Fixes: #1286

Other information:

Hey @ManishMadan2882 the required changes are made can you please check it out.
Feel free to ask for more changes if required :)

Hey @AkashJana18 how is it going?

@dartpain working on it PR will updated soon

Hey @ManishMadan2882  I have made the required changes

Can you specify the kind of app crash on creating new key?
Also can guide me regarding resolving conflict. Like do i need to keep the skeleton reloader or remove it?

Hi @AkashJana18

The app crashes on creating new api key
The error is logged in the console
Uncaught TypeError: Cannot read properties of undefined (reading 'toLowerCase')  at APIKeys.tsx:93:20
We need to keep every change in the main branch including the skeleton.
Thanks.

Hey @ManishMadan2882 conflicts have been resolved.
Can you please check if the apikey error is resolved or not.
Thank you.

Hey @AkashJana18
Do you mind testing these changes before pushing or is there any issue you are facing during set up on your system?
Fails with npm run dev
[plugin:vite:react-babel] /home/manish/Documents/contributors/akash jana/DocsGPT/frontend/src/settings/APIKeys.tsx: Unexpected token, expected "," (57:6)
  60 |       });
/home/manish/Documents/contributors/akash jana/DocsGPT/frontend/src/settings/APIKeys.tsx:57:6
55 |            );
56 |          }
57 |        })
   |        ^
58 |        .catch((error) => {
59 |          console.error(error);

@ManishMadan2882 how do i revert the changes made while resolving conflict. I think i made error while resolving conflict if you can revert the change in APIkeys.tsx i can fix it.
i resolved conflict from github itself (not from vscode).

Follow this


Bug: Jest 20 moduleNameMapper broken

Do you want to request a feature or report a bug?
bug
What is the current behavior?
Module name is not translated :cry:
If the current behavior is a bug, please provide the steps to reproduce and either a repl.it demo through https://repl.it/languages/jest or a minimal repository on GitHub that we can yarn install and yarn test.
git clone https://github.com/orangecms/jest-meteor-demo/
cd jest-meteor-demo
yarn install && yarn test

works so far. Then:
git checkout jest-20
yarn install && yarn test

breaks with the error message:
Cannot find module 'meteor/mdg:validated-method' from 'fooMethods.spec.js'
What is the expected behavior?
Module name should be translated :upside_down_face:
Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.
I'm using Jest 19 and tried upgrading to 20 (see the repo for exact versions) - that broke it.
Node: 7.10.0
npm: v4.2.0
yarn: v0.24.5
OS: GNU/Linux
{
  "transform": {
    "^.+\\.jsx?$": "babel-jest"
  },
  "moduleFileExtensions": [
    "js",
    "jsx"
  ],
  "modulePaths": [
    "<rootDir>/node_modules/",
    "<rootDir>/node_modules/jest-meteor-stubs/lib/"
  ],
  "moduleNameMapper": {
    "^(.*):(.*)$": "$1_$2"
  },
  "unmockedModulePathPatterns": [
    "/^imports\\/.*\\.jsx?$/",
    "/^node_modules/"
  ]
}

I have seen other issues here around the mapper, but couldn't solve it with the suggestions there, including #3565 which could be somehow related. I'm not sure though.

Thanks for the great and simple repro case... I'm able to reproduce it.
Renaming .jestrc to jest.config.js and chaning the contents of it to use module.exports = {...} makes it work
module.exports = {
 // your same config here
};


Make Visualizer#display_candidates mention-type argnostic

This PR assumes a mention has attributes like top, left, top, right, and bottom.
This PR stays as a draft until for example every mention has those attributes.

I was gonna let mentions of any type have top, left, right, bottom attributes, but I'm not working on this direction now.
If and only if an app code adds those attributes to extracted mentions, Visualizer#display_candidates works.
Merge this if this PR is helpful.

Can we fix the test and merge this PR?

Resolved the conflict. Please merge if it looks good.

Can merge this after conflict is resolved.

Let me see what was wrong during OSX test.
There is a chance that the same root cause as https://github.com/HazyResearch/fonduer/pull/347 is affecting here.

https://travis-ci.org/HazyResearch/fonduer/jobs/604026106 was not able to reproduce the error #351 is trying to avoid.
Let me just clean this PR and please merge it once tests are finished.


Make electric anom potencially less laggy on supercrit

About the PR

Possible fix for #16301
This is an attempt to reduce the current lag of the electric anomaly supercrit killing the servers performance and sometimes temporarily breaking audio on the client. In my testing, it seems to at least massively reduce the lag spike.
This does make the supercrit a but less visually impressive, but it still does a big emp effect and still hurts entities close to it.
Make sure to load into a normal game map if you are gonna test it.
Why / Balance

The lag is annoying and at least reducing it should be a good step forward?
Technical details

From what i can understand this reduces the chance for it to pick entities that have power (??) from 20% to now 1% to create a lighting effect to
"Why not create a cap" Unless I did it wrong it still created massive lag
Media

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

:cl: Vasilis

fix: Potentially fix (or hopefully at least reduce) electrical anomaly lagging everyone on supercrit.

not gonna close the issue yet but let's see if this helps.


prefect image specification

@aktech,
I think it would be a good idea to support specifying a prefect image to be used in the qhub config file.
Currently the prefect image specified by qhub is nearly a year old, version 0.12.2, dated June 30, 2020, as seen here.  The latest version as of this writing is 0.14.22, dated June 16, 2021.
Also, the prefect images are python version specific, and we have moved to using python 3.8.
For precedent, recently a similar feature was added to support conda-store image specification in #623, as was dask gateway in this commit.

@brl0 this is a great idea. Like you said there is a PR https://github.com/Quansight/qhub/pull/623/ that already serves as a good template for how to implement this. Adding this to good first issues.

Fixed here: #667


Agency Menu

I think this stopped working as the newest SCNP update. It no longer opens for me.

i just updated the script. should fix it. use a fresh copy to test it and redo the codeplugs after you have confirmed it is working with the new version.

Found another issue, after selecting the codeplug, your camera spins until you press backspace. Thank you for updating it too!! Much love :)


Try to start a docker but it goes wrong

Background
I force rm a container and try to start it again,but it return:
ubuntu@ip-172-31-15-69:~/gocode/src/github.com/lightningnetwork/lnd/docker$ docker-compose run -d --name alice lnd_btc
Starting btcd ... error

ERROR: for btcd  Cannot start service btcd: b'endpoint with name btcd already exists in network docker_default'

ERROR: for btcd  Cannot start service btcd: b'endpoint with name btcd already exists in network docker_default'
ERROR: Encountered errors while bringing up the project.
ubuntu@ip-172-31-15-69:~/gocode/src/github.com/lightningnetwork/lnd/docker$

Your environment
ubuntu@ip-172-31-15-69:~$ uname -a
Linux ip-172-31-15-69 4.4.0-1061-aws #70-Ubuntu SMP Fri May 25 21:47:34 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ip-172-31-15-69:~$ lnd -V
lnd version 0.4.2-beta commit=21841c9f6b065a625e5dcb68c18dfbf3ee25326b
ubuntu@ip-172-31-15-69:~$ lncli -v
lncli version 0.4.2 commit=21841c9f6b065a625e5dcb68c18dfbf3ee25326b
ubuntu@ip-172-31-15-69:~$ btcd -v
Error creating a default config file: open /home/ubuntu/sample-btcd.conf: no such file or directory
unknown flag `v'
Use btcd -h to show usage
ubuntu@ip-172-31-15-69:~$ btcd --version
btcd version 0.12.0-beta
ubuntu@ip-172-31-15-69:~$

Steps to reproduce
1.start a alice docker  container
2.exit forcely and kill the procee
3.restart with "docker-composed"
how to go back ?

@zhouzhou84 I could not reproduce the issue but it looks like one of the containers that docker-compose launched is still attached to the default docker network. You can forcely remove it from there by doing this:
docker network disconnect -f docker_default btcd

If this succeeds, the dontainer is going to be detached from the network and your command will work as the first time. If you find any issue you can also run the following command and then try again:
docker rm -f alice btcd

I hope this helps!

Ok,thank you very much On 2018-08-21 04:53 , Xavi Soler Wrote: @zhouzhou84 I could not reproduce the issue but it looks like one of the containers that docker-compose launched is still attached to the default docker network. You can forcely remove it from there by doing this: docker network disconnect -f docker_default btcd
If this succeeds, the dontainer is going to be detached from the network and your command will work as the first time. If you find any issue you can also run the following command and then try again: docker rm -f alice btcd
I hope this helps! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Seems like this has been resolved? Will re-open if not.


Any chance of Rails 5 for torquebox 3?

We love torquebox 3 and are running an app in production on it, but we'd love it even more if there was support for rails 5 in the future.  Any support for this planned?

I'm also curious about this..


UnboundLocalError: local variable 'logs' referenced before assignment on training with little data

I found an error caused by an attempt of coping training logs from a not yet assigned variable.
The error occurred on my machine (Arch linux, tensorflow v2.2rc2 compiled from source) and I managed to reproduce the error on colab, on a stock environment.
It only happens when the model.fit method is called with very little training / eval data.
The logs variable is assigned inside a for loop that never happens when there is no sufficient data.
The code lives here:
https://github.com/tensorflow/tensorflow/blob/e6e5d6df2ab26620548f35bf2e652b19f6d06652/tensorflow/python/keras/engine/training.py#L793
The notebook gist link for reproducing the bug:
https://colab.research.google.com/gist/naripok/8ce09ec9c3e795b3635a6b1ac11ebd4b/tpu_transformer_model.ipynb

Was able to reproduce the issue with TF v2.2.0-rc2. Please find the gist here. Thanks!

I have the same issue with TF v2.2.0-rc2
Epoch 19/20
18/18 [==============================] - 2s 125ms/step - loss: 0.0035 - accuracy: 1.0000
Epoch 20/20
18/18 [==============================] - 2s 126ms/step - loss: 0.0373 - accuracy: 0.9844

Traceback (most recent call last):
  File "run.py", line 155, in <module>
    main()
  File "run.py", line 120, in main
    accuracy, num_of_classes = train_Full_visible(unique_name)
  File "run.py", line 78, in train_Full_visible
    acc = neuro.train(picdb, train_ids, test_ids, "Full body visible")
  File "/ssd/200410 3rd Try/neuro.py", line 232, in train
    test_loss, test_acc = self.model.evaluate(test_generator, verbose=0)
  File "/home/frank/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/frank/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1028, in evaluate
    logs = tf_utils.to_numpy_or_python_type(logs)
UnboundLocalError: local variable 'logs' referenced before assignment

@Tuxius and others which stuck until this bug is fixed:
I had the same issue  and I found that my validation data set had less samples as the batch_size.  Because I'm working with TFRecords-data-sets which have no meta data about how many records, i.e. samples, are in the data set I check now upfront whether the data set  contains at least batch_size records (samples).
For that I use the following helper functions:
def doesDataSetContainsEnoughDataForBatch(dataset, batch_size):
    return len(list(dataset.take(batch_size).as_numpy_iterator())) == batch_size

def doesDataSetFileContainsEnoughDataForBatch(sampleFileName="", batch_size=100):
    dataset = tf.data.TFRecordDataset(sampleFileName)
    return doesDataSetContainsEnoughDataForBatch(dataset, batch_size=batch_size)

if __name__ == '__main__':
  dataSetFileName="./Samples/validation_data_123.tfrecord"
  if not doesDataSetFileContainsEnoughDataForBatch(dataSetFileName,batch_size=100):
    raise Exception(f"Data set file {dataSetFileName} doesn't contain enough data")

  # Now open the data set a second time knowing you have enough data
  # and use it ....
  """
  trainDataset = tf.data.TFRecordDataset(dataSetFileName)

  model.fit ( trainDataset ...

  """

Please keep in mind that by using the first helper function directly ,
i.e. doesDataSetContainsEnoughDataForBatch,  you already read batch_size samples from the data set. So you should recreate the data set after the check.
By using the second helper function  you just lose some execution time upfront.
If you don't use TFRecord data sets you might have a similar  issue. But then it is also a good idea to check upfront whether you have enough data for at least one batch.

@Tuxius and others which stuck until this bug is fixed:
I had the same issue and I found that my validation data set had less samples as the batch_size. Because I'm working with TFRecords-data-sets which have no meta data about how many records, i.e. samples, are in the data set I check now upfront whether the data set contains at least batch_size records (samples).
For that I use the following helper functions:
def doesDataSetContainsEnoughDataForBatch(dataset, batch_size):
    return len(list(dataset.take(batch_size).as_numpy_iterator())) == batch_size

def doesDataSetFileContainsEnoughDataForBatch(sampleFileName="", batch_size=100):
    dataset = tf.data.TFRecordDataset(sampleFileName)
    return doesDataSetContainsEnoughDataForBatch(dataset, batch_size=batch_size)

if __name__ == '__main__':
  dataSetFileName="./Samples/validation_data_123.tfrecord"
  if not doesDataSetFileContainsEnoughDataForBatch(dataSetFileName,batch_size=100):
    raise Exception(f"Data set file {dataSetFileName} doesn't contain enough data")

  # Now open the data set a second time knowing you have enough data
  # and use it ....
  """
  trainDataset = tf.data.TFRecordDataset(dataSetFileName)

  model.fit ( trainDataset ...

  """

Please keep in mind that by using the first helper function directly ,
i.e. doesDataSetContainsEnoughDataForBatch, you already read batch_size samples from the data set. So you should recreate the data set after the check.
By using the second helper function you just lose some execution time upfront.
If you don't use TFRecord data sets you might have a similar issue. But then it is also a good idea to check upfront whether you have enough data for at least one batch.

I added more data. It works well!
Thank you!

Hi there, I change the batch_size to smaller so that it could be more evenly divided during training.
For example, when you have 300 training samples, do not use batch_size=128, try batch_size =10 or batch_size=20 something, it works for me.

The issue is still reproducible with 2.2.0rc4, will test with 2.2.0 release.

2.2 is still reproducible for me

2.2 is still reproducible indeed

Are there any other workarounds known yet? The batch size one did not work out for me.

Same problem

same problem: Is there way ro fix the issue?

Same problem, the root cause for this issue is that the training cannot perform a single step because your dataset is not large enough to fill one training iteration.
Anyway, there is still a bug here, this is not the expected behavior.

Same problem, the root cause for this issue is that the training cannot perform a single step because your dataset is not large enough to fill one training iteration.
Anyway, there is still a bug here, this is not the expected behavior.

Yes. You are right. When i increased training data the error doesnot occure.

I have also same error arise , even increase the training dataset its not solved. Again i got the same error. kindly share me the solution if anyone knows about this.
Thank you

I have also same error arise , even increase the training dataset its not solved. Again i got the same error. kindly share me the solution if anyone knows about this.
Thank you

What is the size of the dataset you are using?

512, 512 size and 247 images and masks

I solved my particular issue setting the batch_size very small, in my case batch_size = 1. I confirm that this happens with small datasets.

yes  sir, i have also reduced batchsize as 1, but i dont know the reason why the same error arise.

I had the same error with using model.fit it turns out the validation_steps or steps_per_epoch was zero

same problem also. is there a workaround?

same problem during training BiT. Batch size 128, Steps_per_epoch = 100

Working with a small dataset
changed batch_size = 64 - worked

This error also raise when len(traindata) or len(validdata) is 0.

I found this error while running the training part of vgg16 model
def vgg16_model( num_classes=None):
model = VGG16(weights='/kaggle/input/keras-pretrained-models/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5', include_top=False, input_shape=(224, 224, 3))
x=Flatten()(model.output)
output=Dense(num_classes,activation='softmax')(x)
model=Model(model.input,output)
return model

vgg_conv=vgg16_model(6)
def kappa_score(y_true, y_pred):
y_true=tf.math.argmax(y_true)
y_pred=tf.math.argmax(y_pred)
return tf.compat.v1.py_func(cohen_kappa_score ,(y_true, y_pred),tf.double)

opt = SGD(lr=0.001)
vgg_conv.compile(loss='categorical_crossentropy',optimizer=opt,metrics=['accuracy',kappa_score])
nb_epochs = 3
batch_size = 16
nb_train_steps = train.shape[0]//batch_size
nb_val_steps=validation.shape[0]//batch_size
print("Number of training and validation steps: {} and {}".format(nb_train_steps,nb_val_steps))
check_point = ModelCheckpoint('./model.h5',monitor='val_loss',verbose=True, save_best_only=True, save_weights_only=True)
early_stop = EarlyStopping(monitor='val_loss',patience=25,verbose=True)
callbacks = [check_point,early_stop]
vgg_conv.fit_generator(
train_generator,
verbose=2,
steps_per_epoch=nb_train_steps,
epochs=nb_epochs,
validation_data=validation_generator,
validation_steps=nb_val_steps,
callbacks=callbacks,
use_multiprocessing=True)
Found 16 non-validated image filenames belonging to 6 classes.
Found 4 non-validated image filenames belonging to 2 classes.
Number of training and validation steps: 1 and 0
Epoch 1/3

UnboundLocalError                         Traceback (most recent call last)
 in
63     validation_steps=nb_val_steps,
64     callbacks=callbacks,
---> 65     use_multiprocessing=True)
/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
322               'in a future version' if date is None else ('after %s' % date),
323               instructions)
--> 324       return func(*args, **kwargs)
325     return tf_decorator.make_decorator(
326         func, new_func, 'deprecated',
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
1477         use_multiprocessing=use_multiprocessing,
1478         shuffle=shuffle,
-> 1479         initial_epoch=initial_epoch)
1480
1481   @deprecation.deprecated(
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
64   def _method_wrapper(self, *args, **kwargs):
65     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
---> 66       return method(self, *args, **kwargs)
67
68     # Running inside run_distribute_coordinator already.
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
870               workers=workers,
871               use_multiprocessing=use_multiprocessing,
--> 872               return_dict=True)
873           val_logs = {'val_' + name: val for name, val in val_logs.items()}
874           epoch_logs.update(val_logs)
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
64   def _method_wrapper(self, *args, **kwargs):
65     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
---> 66       return method(self, *args, **kwargs)
67
68     # Running inside run_distribute_coordinator already.
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in evaluate(self, x, y, batch_size, verbose, sample_weight, steps, callbacks, max_queue_size, workers, use_multiprocessing, return_dict)
1089       callbacks.on_test_end()
1090
-> 1091       logs = tf_utils.to_numpy_or_python_type(logs)
1092       if return_dict:
1093         return logs
UnboundLocalError: local variable 'logs' referenced before assignment

Can we add tag 2.3 for this issue?

Can we add tag 2.3 for this issue?
I am using tensorflow 2.2.0

I am surprised there is no self-contained code example reproducing this, so here is one:
import numpy as np
from tensorflow import keras
model = keras.models.Sequential([keras.layers.Dense(1, input_shape=(1,))])
model.compile(loss="mse", optimizer="adam")
model.fit(x=np.ones((1, 1)), y=np.ones((1, 1)), validation_split=0.5)

This is reproducible in TF 2.3. I am using batch_size of 1 and no validation data.

Encountered this error, myself, and thought I'd share my debugging in case others would benefit in some way.
Tracing the calls and returns, it seems that inside fit, a local variable logs is first defined inside a for loop iterating over DataHandler.steps(), which, in many cases, is a number dependent on the cardinality of the Dataset. If the Dataset cardinality is zero, then logs will never be created, resulting in the UnboundLocalError we're all getting.
The cardinality of my Dataset was being returned as 0 because I had created it from a .take() called on a BatchDataset -- which was an artifact of a previous iteration of my code.
After cleaning the code (in my case, calling take on a Dataset that had not been batched), my Dataset.cardinality() returned a nonzero value, and the error was resolved.

when turn off validation_data, it works! tf.version == '2.2.0'
logistic_regression.fit(train_dense_x,
          train_label, epochs=100, batch_size=256,
#           validation_data=(val_dense_x, val_label),
          callbacks=[tbCallBack])

check number of your training data. It might equal to zero

Looks like this happens when size of the training data is <= batch size

check number of your training data. It might equal to zero

In my case, I read my tfrecords incorrectly but didn't get any warnings or errors. After reading the record, you can try printing its size as a sanity check.

@naripok the original issue of "UnboundLocalError: local variable 'logs' referenced before assignment" is no longer with the latest tf-nightly. Can you please confirm and if so close the issue ?

Sorry for the delay,
Confirmed.
Thanks!

had less samples as the batch_size

The same situation can occur if the train-set has less number of examples than the batch size. In my case my train-set's directory was wrongly given.

check number of your training data. It might equal to zero

Thank you!


logout session clean

Gitea version (or commit ref): Gitea version 1.6.1 built with: bindata, sqlite
Git version: git version <IP_ADDRESS>
Operating system: Centos 7.4
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[ ] MSSQL
[x] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[x] No
[ ] Not relevant

Log gist:

Description
Now I use nginx to forward requests to gitea, the configuration as follows:
server
{
    listen 80;
    server_name xxx.cn; # this is sample domain
    index index.php index.html index.htm default.php default.htm default.html;
    root /www/wwwroot/gitea;
    location /
    {
        proxy_pass http://<IP_ADDRESS>:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;

        expires 12h;
    }
}

When I logged in and clicked the logout button, nginx caches requests( Status Code: 302 Found (from disk cache) ), cause not to receive the correct response header, the 'set-cookie' response header miss so I can still operate normally.
When I look at the folder 'data/sessions', user session still exists, is this normal or should the session be cleared when the user exits?
Because my English level is limited, I use Google Translator. If there is something unclear, please tell me that I am willing to assist in the recurrence of the problem.
...

So I think this is normal.  There's certainly no call to destory (sic) the session in the code. I think however there's some clean up process that will remove old sessions intermittently but I could be wrong in that.


Decompression failed

Hello,
I am receiving this error:
    file = deflate.gzip_decompress(file.read())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
deflate.DeflateError: Decompression failed

Reproduce:
pip install deflate

Here is the code:
    with open(file_path, "rb") as file:
        file = deflate.gzip_decompress(file.read())
    file = BytesIO(file)

The Python version is 3.12 and Ubuntu 22.04.
File is .gz.
Thank you.

Does it work with Python's built-in gzip.decompress function? My first thought is the GZ file is not actually GZip, or corrupt somehow, but hard to know for sure.

Hello,
Thank you for the response.
The file is a warc.gz file.
If it is helpful for diagnosis. The file being tested can be obtained here: wget https://data.commoncrawl.org/crawl-data/CC-MAIN-2018-17/segments/1524125937193.1/warc/CC-MAIN-20180420081400-20180420101400-00000.warc.gz

FYI, I was curious, and yes, it does work with gzip.decompress. It takes a long time (the zipped file is 888 MB), but it does work. While the failure is pretty much instant with deflate.
$ docker run --rm -it quay.io/pypa/musllinux_1_2_x86_64
# wget https://data.commoncrawl.org/crawl-data/CC-MAIN-2018-17/segments/1524125937193.1/warc/CC-MAIN-20180420081400-20180420101400-00000.warc.gz
# python3.12 -m pip install deflate
# python3.12

>>> import deflate
>>> import gzip
>>> file_path = "CC-MAIN-20180420081400-20180420101400-00000.warc.gz"
>>> with open(file_path, "rb") as file:
...     file = deflate.gzip_decompress(file.read())
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
deflate.DeflateError: Decompression failed
>>> with open(file_path, "rb") as file:
...     file = gzip.decompress(file.read())
...

(Technically, I gave up waiting on that, but gzip.open(file_path).read() did return eventually when I tried it originally).

The built-in archive utility on macOS also chokes on this file. But plain old gunzip works, and it expands to a 3.7 GB file (fairly quickly). libdeflate is probably failing fast due to one of the checks in https://github.com/ebiggers/libdeflate/blob/master/lib/gzip_decompress.c but I'm not sure I want to go digging into which specifically, as it's probably out of scope for this library.
In any case, libdeflate is definitely the wrong tool for files of this size - see https://github.com/ebiggers/libdeflate?tab=readme-ov-file#api for comments from the author.

FYI, I was curious, and yes, it does work with gzip.decompress. It takes a long time (the zipped file is 888 MB), but it does work. While the failure is pretty much instant with deflate.
$ docker run --rm -it quay.io/pypa/musllinux_1_2_x86_64
# wget https://data.commoncrawl.org/crawl-data/CC-MAIN-2018-17/segments/1524125937193.1/warc/CC-MAIN-20180420081400-20180420101400-00000.warc.gz
# python3.12 -m pip install deflate
# python3.12

>>> import deflate
>>> import gzip
>>> file_path = "CC-MAIN-20180420081400-20180420101400-00000.warc.gz"
>>> with open(file_path, "rb") as file:
...     file = deflate.gzip_decompress(file.read())
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
deflate.DeflateError: Decompression failed
>>> with open(file_path, "rb") as file:
...     file = gzip.decompress(file.read())
...

(Technically, I gave up waiting on that, but gzip.open(file_path).read() did return eventually when I tried it originally).

Hi @henryiii,
The file will work with gzip.decompress. I tested with a variety of warc.gz files. Each time it was incredibly slow.
Doing something standard like:
with open("my warc.gz", "rb"):
    file = io.BtyesIO(file.read())

Works much faster to decompress and open.

The built-in archive utility on macOS also chokes on this file. But plain old gunzip works, and it expands to a 3.7 GB file (fairly quickly). libdeflate is probably failing fast due to one of the checks in https://github.com/ebiggers/libdeflate/blob/master/lib/gzip_decompress.c but I'm not sure I want to go digging into which specifically, as it's probably out of scope for this library.
In any case, libdeflate is definitely the wrong tool for files of this size - see https://github.com/ebiggers/libdeflate?tab=readme-ov-file#api for comments from the author.

Hi @dcwatson,
Thank you for the additional information. Given that this is performance-sensitive at scale, every additional second for decompression adds up. So having something that can decompress on average faster is quite meaningful. It looks like deflate may not be the best for this case then given the size of the .gz files.
I have been benchmarking a few other decompression libraries.
@dcwatson @henryiii Do either of you have thoughts on libraries worth testing?
Thank you.

I'd probbaly look at zlib-ng - there are python bindings here
Not an endorsement, as I've never used either, but it seems like zlib-ng is significantly faster than vanilla zlib and does streaming decompression.

I'd probbaly look at zlib-ng - there are python bindings here
Not an endorsement, as I've never used either, but it seems like zlib-ng is significantly faster than vanilla zlib and does streaming decompression.

Thank you for the information.
I will add this to the benchmark as well.

I don't actually know anything about decompression[^1], I just helped with the packaging. :)
[^1]: I've technically done a little research in algorithms a long time ago, but not familiar with libraries

I don't actually know anything about decompression1, I just helped with the packaging. :)
Footnotes

I've technically done a little research in algorithms a long time ago, but not familiar with libraries ↩

Pybind is an awesome library so we definitely appreciate all your work.


iOS Youtube embeded Play Button

If you scroll down to the video at the bottom on https://theissue.fuelthemes.net/
The Youtube Video is not playable, meanwhile Vimeo works without any issues.

if you add attr «playsinline=1» in video url like this :
src="https://www.youtube.com/watch?v=xxxxxxxxxxx&playsinline=1"
the video is playable.
But I can't put her on pause....

Hi @anteksiler,
I just resolved this issue for one of my clients.  The solution for me was to use the sharing video url's generated by clicking the "share" button on a video page rather than just copying the url in browser.
This work: https://youtu.be/xxxxxxxx
This doesn't: https://www.youtube.com/watch?v=xxxxxxxx


fix(module): fix forth assignment not being populated

:tada: This PR is included in version 1.4.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add JPMS module name or descriptor

What happened?
For the Java Platform Module System, square/javapoet defines an Automatic-Module-Name (see pom.xml config).
It looks like this fork here neither defines an Automatic-Module-Name, nor does it have a module-info.class. This makes it difficult to use in a modular project.
What did you want to happen?
This project should either define an Automatic-Module-Name in the manifest, or contain a module-info.class (respectively META-INF/versions/9/module-info.class and in manifest Multi-Release: true).

I think the project should at least just follow the way the original repo did, and simply define the "Automatic-Module-Name" in the pom/gradle:
ext.moduleName = "com.palantir.javapoet"

jar {
    inputs.property("moduleName", moduleName)

    manifest {
       attributes  'Automatic-Module-Name': moduleName
   }
}

I would like to provide the PR, but I am not sure, which "build.gradle" should be used. Is it the one inside the "javapoet" folder or the root one?

I believe it would go into this one: https://github.com/palantir/javapoet/blob/develop/javapoet/build.gradle

Submitted a PR. It now adds the entry into the manifest properly:

Although I am not sure whether this will work, when built as an artifact for maven. Due to the unfamiliar build system ("gradlew build" builds two jar files: empty root jar file and the "javapoet" jar file), I only tested by building locally and linking to my other gradle, and it seems to work with the actual "javapoet" jar file, and considering that the root jar file is literally empty, I'm pretty sure that it should work properly, even when built in maven.

Thanks for the contribution! I expect it to work, but you should be able to merge within ~10 minutes or so when the artifact hits maven central :-)

0.4.0 just hit the central; requires com.palantir.javapoet; in module-info now works properly, and the imports work as intended with JPMS projects without any warnings.


guthub gradlew build fails when using filetree or flatDir

Expected Behavior
When running a local build using gradlew it resolves all folders that contain dependencies from the local file system when using fileTree or flatDir.
The same should happen when running it from a github action runner. It doesn't.
Current Behavior
Run ls -al /home/runner/work/game-client/game-client/sync/network
total 28
drwxr-xr-x 2 runner docker 4096 Aug 27 04:45 .
drwxr-xr-x 4 runner docker 4096 Aug 27 04:45 ..
-rw-r--r-- 1 runner docker 5392 Aug 27 04:45 common-game-0.4.0.jar
-rw-r--r-- 1 runner docker 2355 Aug 27 04:45 common-host-0.3.0.jar
-rw-r--r-- 1 runner docker 7003 Aug 27 04:45 util-java-0.3.0.jar

* What went wrong:
> Task :compileJava FAILED
Execution failed for task ':compileJava'.
:compileJava (Thread[Daemon worker,5,main]) completed. Took 4.789 secs.
> Could not resolve all files for configuration ':compileClasspath'.
   > Could not find :common-game:0.4.0.
     Searched in the following locations:
       - file:/home/runner/work/game-client/game-client/sync/network/common-game-0.4.0.jar
       - file:/home/runner/work/game-client/game-client/sync/network/common-game.jar
     Required by:
         project :

Context
Cannot resolve local file depends from project root on github using fileTree or flatDir.
Steps to Reproduce
Add a jar to a folder in root of project with a gradle dependency to that jar or jars by using
    repositories {
        //Common network files dir
        flatDir {
            dirs "$rootProject.projectDir/sync/network"
        }
...
dependencies {
...
    //Common networking jars
    implementation ":common-game:0.4.0"

or just use
implementation fileTree(dir: "$rootProject.projectDir/sync/network", include: ['*.jar'])

use the github actions checkout and run gradlew.
See this project workflow,
https://github.com/mitm001/Packit/blob/master/.github/workflows/gradle.yml
it has an extra build command
./gradlew build

to show the exact problem in logs.
Using the list command shows the files exist before running gradle.
Your Environment
Github workflow project using gradle 6.6.1, see link above and add jar to root and dependency in gradle to that jar.

Closing the issue as I figured it out…
If you use project.sync {} locally to move jars between projects, and push to github workflow, sync cant find the folders it was syncing.
Sync will still delete the contents of the folders you were moving files into though. So depends failed as they were being deleted.
I will put the from/into inside a task instead, make sure the folders exists and call the task if they do.


Pytorch Lightning useful?

Motivation for the feature

I came across Pytorch Lightning, which seems to have a few useful features, including:

multiple-GPUs, TPUs, CPUs and even in 16-bit precision without changing your code!
implementation device agnostic
automates things like tensorboard logging

The purpose of this issue is to gather the team's feedback to see if it would be relevant to include Pytorch Lightning in our ecosystem.

I have used PyTorch Lightning (PL) before and it really makes everything a lot simpler. Most importantly, PL-based code is easily reproducible in that the standard PyTorch code is divided into different chunks (i.e. training, validation, optimization, dataset creation) as can be seen here. This visualization also shows why it is suited for quick prototyping.
As I understand it, since PL is a higher-level library built on top of PyTorch, the codebase should remain the same, however, one of the major tasks would be to put the correct snippets of code into the relevant functions and classes as defined by PL.

How does this intersect with onnxruntime? torch is an extremely large dependency and it causes headaches in SCT (https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/3396). We were hoping to have you move ivadomed towards onnxruntime for inference (https://github.com/ivadomed/ivadomed/issues/232 / https://github.com/ivadomed/ivadomed/issues/795). Would Lightning re-entrench torch?

I haven't worked with onxx and came across the issue when I was doing 2D segmentation tutorial.  In my case, I had to disable (read comment out) the onxx part to get it running. As for torch and pytorch lightning , I have used both (may be not as much as Naga). Later one is, as @naga-karthik said, a high level thing which abstracts a lot of stuff in pytorch. From my experience, the main difference is seen in training and testing code which is relatively smaller in lightning and hence much more readable.
This article is a decent place to compare them and the link Naga gave it also very good!
Having said that, though I loved pytorch lightning, I found it hard when debugging issues. Because it is abstract, a lot of things may not be necessarily clear. This link is a little old but I share the same feeling. Having said that, it's been 8 months since I used lightning and it may have progressed.
For me the biggest reason of using pytorch was its forum (supppper helpful and active) -- especially these two people Tom (https://discuss.pytorch.org/u/tom) and @ptrblck(https://discuss.pytorch.org/u/ptrblck/summary).  At times, I just tagged these people and got my answers within 24 hours or even earlier. I wasn't stuck. Lightning forum exists but I haven't used extensively.
It wouldn't be best to move to lightning completely without everyone having hands on knowledge about it :smile:. It's not what people come prepared by default, though they may have dabbled into it. It will affect collaboration.

I never used PyTorch Lightning. It seems to allow to remove some common boilerplate code. However, here are some point to check before trying it:

[ ] What would be the migrating effort required? (Seems "epic" to me)
[ ] Could the migration be broken down in seamless sub-tasks?
[ ] Should we change the function signatures? (For example, device and cuda_available would probably not be used since the data-device management happens under the hood when using PL.)
[ ] Is the gain worth the effort? (If I would have to start a new project, I would use it right away, but now we have a whole project to migrate. If it can be done by steps it could be conceivable, otherwise I'm not sure.)

Is it still a high priority?

Is it still a high priority?

good question-- @naga-karthik @mariehbourget @uzaymacar what do you think?

Personally, I am all in-favour for PyTorch Lightning (PL). All my personal projects involve the usage of PL, mainly because, PL makes it really easy to do multi-GPU training, log on wandb, convert to 16-bit precision, etc. all of which require some extra code to be added around the training loop.
That said, I do understand that it might require a significant restructuring of the existing codebase. Hence, to save everyone's time, I would be interested to see if we can add PL functionality into the existing code without modifying the legacy codebase. I am not sure what this actually means (it still a bit fuzzy in my mind), however, I think there can be some solution here because PL is basically built on top of PyTorch. It would be great if we could discuss this in the next meeting!

Sorry for the misclick 😅

On the release note of PL version 1.6, they state

Following our 4 PyTorch release window, this release supports PyTorch 1.8 to 1.11. Support for PyTorch 1.7 has been removed.
So, I understand that they support the 4 last main PyTorch releases at the release time of the main version. Therefore, the last version of PL supports the PyTorch version we're currently using (i.e v1.8.1).

Note: PL 1.6 no more supports Python 3.6 which is still supported by ivadomed.

Note: PL 1.6 no more supports Python 3.6 which is still supported by ivadomed.

We haven't updated yet but ivadomed indeed should drop support for Python 3.6 due to its end of life on 23 Dec 2021


Test package setup

Docs Build status updates of commit 2eb7e96:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error]

Cannot sync git repo to specified commit because branch testPackageSetup has been deleted or has been force pushed: fatal: Couldn't find remote ref testPackageSetup

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Fix touchscreen input related functions on Android

Fix display -> screen coordinate conversion for android platform and move it to the platform event handling code, simplifying GetTouchPosition() function implementation.
Fixes previous behavior on Android which is reporting touchscreen input coordinates as a number in range from 0 to 1 to reporting values in screen coordinates.
Tested with native resolution and upscaling. Downscaling currently seems to have another bug somewhere deeper in rendering setup.

@deniska Thanks for the review!


upgrade to selenium-java:3.7.1

Proposed changes
Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request.
If it fixes a bug or resolves a feature request, be sure to link to that issue.
Checklist

[X] Checkstyle and unit tests pass locally with my changes by running gradle check chrome htmlunit command
[] I have added tests that prove my fix is effective or that my feature works
[] I have added necessary documentation (if appropriate)

Coverage decreased (-0.07%) to 64.281% when pulling 1e0d6664b616302d963d8ff6d18f00ed8b733738 on upgrade-to-selenium371 into 2d40bf1996f0fd61a375e6f5dc758f94eaf8e605 on master.

Codecov Report

Merging #635 into master will decrease coverage by 0.1%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #635      +/-   ##
============================================
- Coverage     60.53%   60.42%   -0.11%
+ Complexity      772      771       -1
============================================
  Files           148      148
  Lines          2752     2752
  Branches        272      272
============================================
- Hits           1666     1663       -3
- Misses          981      983       +2
- Partials        105      106       +1

Impacted Files
Coverage Δ
Complexity Δ

...e/selenide/impl/WebDriverThreadLocalContainer.java
80.43% <0%> (-2.18%)
30% <0%> (-1%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d40bf1...1e0d666. Read the comment docs.


Typo : CRITIAL VM ERROR

The error message appearing for some of yolol errors has a typo (see screenshot below).

When you get a CRITIAL VM ERROR there should also be some info in the console that Yodine opens with, I have added an extra part to the error to say that.
I will fix this now (I already reproduced)


Bug 2053582: Track static pod lifecycle

Improves MissingStaticPod controller to read a static pod revision from the mirror pod.
This approach has the advantage over the previous one because is able to detect potential issues during the bootstrap phase. Previously all failures would be ignored because we skipped CurrentRevision == 0

/lgtm

/cherry-pick release-4.10


💡[FEATURE]:  Add Slack Logo and link in the README

We created a Slack channel recently for our project to interact contributors with mentors. Need to update the README with its logo and the channel link:

https://join.slack.com/t/newworkspace-ley5153/shared_invite/zt-2isvflg78-tMexsn50mxdwY4BEJ_W11g

Contributions Welcome 🚀

@varshith257 Please assign this issue to me, I will try to fix this issue within 24 hours.

@varshith257 Please assign this issue to me, I will try to fix this issue within 24 hours.

@MrCodYrohit It's a small fix can be implemented within 10 minutes. @Arpcoder will work on it and thanks for your enthusiasm to contribute for this project

@varshith257 Please change the syntax of labels from 'GSSOC 24' to 'gsoc' and 'Level 1' to 'level1' as per the requirement under GSSoC. Thank you.

@varshith257 Please change the syntax of labels from 'GSSOC 24' to 'gsoc' and 'Level 1' to 'level1' as per the requirement under GSSoC. Thank you.

Ok, I will add it. Thanks for reminding

@varshith257 Now, please reassign me, I will fix it in 10 mins.


Debugging features

Totally understand if you don't want to merge this, it's a potentially large departure from your vision.
Demo of usage at http://youtu.be/r2NE0SkzWXA in which I automatically filter down SMB RAM to interesting values.

Cool! Thanks for adding this! I had rudimentary debugging support early on that I used to bang through some issues but nothing that was terribly useful or worth keeping around.


Feature: Theme Feature in settings

Describe the feature
The theme feature in this app allows users to personalize the appearance of the app by choosing between different themes, such as light and dark themes. Themes define the visual style of various UI elements like buttons, text, backgrounds, and more.

assigned @andoriyaprashant

Hey @savitore  Could you assign this issue to me?

assigned @sameermansoori1


Enable TLS options to be controlled by hostname via a callback

https://github.com/dotnet/runtime/issues/31097

Addresed

Re-opening as this still needs to be reacted to in Kestrel.
https://github.com/dotnet/aspnetcore/issues/20981

This has now been addressed in Kestrel.


Oneclick capabilities

This will require some frontend work and some sort of plugin communication
From python, one possible way to do this is add an arg, which supports a string or list of strings oneclick=["column1","column2",...]

This might actually be possible with current architecture by sending a list of values in the datamapping such as
"AAPL": 0,
"GOOG": 1
}```
where the number is the trace within the data that the specific category is associated with


Translation

To translate this I need to create ja.json in this dir, right?

Yep, but there are also a couple of more steps. I think the best approach is to follow the Docusarus instructions:
https://docusaurus.io/docs/en/translation.html#translating-your-existing-docs
Happy to help you with this!

@cvalenzuela Thanks.
I created ja.json and translated it. I will ask you some questions on that.


Test build CI integration should post results in the PR even if one platform fails

Problem
With the help of @staszekscp, we recently built a CI integration that will run test builds on all PRs, and can be manually dispatched by authorized actors.
When testing this out with https://github.com/Expensify/App/pull/13767, I noticed that the build succeeded on all platforms except one. However, upon completion there was no comment posted in the PR, so it's not easy (or at least not obvious how) to test the platforms that did succeed.
Solution
Make the final step that posts the comment run even if one or more platforms fail. A similar example can be seen here, and the resulting comment should be tailored to clearly show for which platforms the build was successful and for which it was not. This is useful information to a PR author and also can help others proactively test the platforms for which the build was successful.

Slack thread: https://github.com/Expensify/App/issues/13784

Sure, I'll take care of that!

PR is in review.

Just left another review on the PR.

This was done a couple weeks ago


The Bellman's Speach by Bajka unsupported.

Check if issue with swaglyrics or whether song lyrics unavailable on Genius.

stripper -> Bajka-The-Bellmans-Speach

speech spelt wrong on genius


dataset

Can you publish your dataset?

Dear @123456ZF,
The dataset can be downloaded from here.
I modified the torchvision.datasets.UCF101 for this repository.

@daniel-code Thank you


Which RBAC role require to change the size of VM

Please consider to add to this document about which RBAC role user should have to perform the resizing

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7c823b1f-688f-e1b9-1590-92d12c28b138
Version Independent ID: ac483660-9046-51b6-b678-4e6f5cd15e74
Content: Use PowerShell to resize a Windows VM in Azure
Content Source: articles/virtual-machines/windows/resize-vm.md
Service: virtual-machines-windows
GitHub Login: @cynthn
Microsoft Alias: cynthn

Thanks for the feedback! We are currently investigating and will update you shortly.

Thanks for the feedback @KeiroKamioka0000
Generally it is assumed the owner or at least a contributor to the subscription would be doing this commands so we don't call out the requirements here.
You can find them in the RBAC documentation though
https://docs.microsoft.com/en-us/azure/role-based-access-control/overview
At this time we have opted to not update the doc with this info. However your comment will remain on the doc and if others feel the same they can comment and we can always reconsider in the future.


feat: darkMode for user profile has been done

Issue #109 📐

I worked on feature #109

Guidelines 🔐
I accept the fact that I have followed the guidelines and have not copied the codes from around the internet
Issue to be closed 🛅

My pull request closes #109

Screenshots 📷
Here are the video of changes that I have made 🔽
https://github.com/warmachine028/memories/assets/110036553/6bad3cf4-f549-4dcf-93db-7baabedbd308

Pretty quick, quite impressive work. I'll review it

Sure

@warmachine028 please review the code in which I replaced the inline styles with classes.

Could you please specify the location of the comments and the card?
it looks ok here

https://github.com/warmachine028/memories/tree/main/client/src/components/PostDetails/CommentSection
https://github.com/warmachine028/memories/tree/main/client/src/components/PostDetails/CommentSection

card text has been changed to white

comment title text also changed

Added border to card

Also resolve the conflicts too

No worries I have resolved the merge conflict. Happy hacktoberfest 🎉


Hi I need to extract the documents per topic after doing a HDP inference

Hi I need to extract the documents per inferred topic using HDP, how can I do this? Thank you.

Hello @RUMCS, sorry for late feedback.
What you want is to pick documents based on its topic, isn't it?
Since, in topic models including HDP, every document contains many topics, it is ambiguous to decide which document is on a particular topic.
Let's briefly define 'relatedness' to certain topics of a document as follows:

A document is related to topic X if topic X is in the top 'n' among the distribution of topics in the document.

In this case, the documents related to each topic can be extracted with following codes.
# we use only top-1 topic for relatedness
top_n = 1

# initialize model with appropriate parameters
mdl = tp.HDPModel()

#
# ... add_doc and training ...
#

# prepare documents for inference
docs = []
for line in open('some_docs.txt'):
  docs.append(mdl.make_doc(line.strip().split()))

# let's infer
dist, ll = mdl.infer(docs)

# extract documents related to each topic
docs_per_topic = [[] for _ in range(mdl.k)]
for doc in docs:
  for topic, weight in doc.get_topics(top_n=top_n):
    docs_per_topic[topic].append(doc)

# print documents related to each topic
for k in range(mdl.k):
  if not mdl.is_live_topic(k): continue
  print('Topic #{} {}'.format(k, ' '.join(word for word, _ in mdl.get_topic_words(top_n=5))))
  for doc in docs_per_topic[k]:
    print(doc)
  print()

You can control the extracted documents by chaning top_n value or adding another condition such as weight threshold of topics.

@fuaduae Oops, sorry for late answer.
The code in the above comment has typo errors.
mdl.get_topic_words(top_n=5) should be corrected to mdl.get_topic_words(k, top_n=5).
for k in range(mdl.k):
  if not mdl.is_live_topic(k): continue
  print('Topic #{} {}'.format(k, ' '.join(word for word, _ in mdl.get_topic_words(k, top_n=5))))
  for doc in docs_per_topic[k]:
    print(doc)
  print()

I am trying to get topic probabilities for each document after training an HDP model but the kernel keeps dying. I only have 12,000 documents and I tried scoring in 100 document chunks but still the same issue. I tried changing together parameter to False but it didn't make any difference either.
I was able to generate topic probabilities on the same set of documents with no issues using  LDA model.
for i in range(0, len(texts), 100):     doc_inst = [model.make_doc(text) for text in texts[i:i+100]]     review_topics = model.infer(doc_inst, together = True)
Any suggestions?

Hi @Jurgita-DS,
Could you share the version of your python and tomotopy where the problem occurs?

Hi @bab2min,
I am using python version 3.7.4 and tomotopy  0.10.2.
The issue seems to be somehow related with scoring certain document instances, however, I cannot see anything wrong with those documents.
texts is a list of documents where each document is a list of tokens, e.g. [[word1, word2], [word3, word4, word5]].
If I, for example, exclude the 8000th document the below code works fine.
doc_inst = [model.make_doc(text) for text in texts[8001:8500]]
doc_topics = model.infer(doc_inst, together = True)

Also if I only include the 8000th document the code runs with no issue.
doc_inst = [model.make_doc(text) for text in texts[8000]]
doc_topics = model.infer(doc_inst, together = True)

However, neither of the below two options work. Both cause the kernel to die.
# Generate topics for documents 8000 and 8001st
doc_inst = [model.make_doc(text) for text in texts[8000:8002]]
doc_topics = model.infer(doc_inst, together = True)

# Generate topics for document 8000th
doc_inst = [model.make_doc(text) for text in texts[8000:8001]]
doc_topics = model.infer(doc_inst, together = True)

I cannot see anything wrong with the document 8000. It contains a small number of words which all look fine to me.
I have also tried converting the HDP model into LDA, however, got the below error.
'there is a document with uid =  already'
This is the code I used:
model_conv = model.convert_to_lda(topic_threshold = 0)
I am not sure if it is related with the first issue of kernel dying or it's a separate one.
I have checked that there are no empty documents in the corpus (all contain at least one token), and no strange tokens (e.g. space).
I hope this helps.
I am working with the data that I cannot be shared publicly so I am not able to share the data and the model that is causing the issue. I tried running the same code on the open source data but wasn't able to reproduce the error.

@Jurgita-DS
Thank you for sharing details.
It seems that the combination of the 8000th document's content and the trained HDPModel cause a crash.
Probably there is a potential flaw in the code on the current infer method of HDPModel. As a result of writing the test code as follows in the same environment as the one you shared with, we found that a crash occurs in the infer function with some probability.
Based on your situation , I wrote the following test code.
When I ran it in the same environment that you shared with me, I confirmed that a crash occurs in the infer method sometimes.
while 1:
  import tomotopy as tp
  mdl = tp.HDPModel(min_cf=2)
  mdl.add_doc("a b c d e f".split())
  mdl.add_doc("1 2 3 d e f".split())
  mdl.add_doc("c d".split())
  mdl.add_doc("3 f".split())

  mdl.train(1000)

  doc1 = mdl.make_doc("a b c".split())
  doc2 = mdl.make_doc("b c d e".split())

  mdl.infer([doc1, doc2])

Also, the problem where convert_to_lda raises there is a document with uid = already exception seems another bug.
I will try to determine the cause of two bugs you reported and fix them.
Thank you for your reporting again!

@Jurgita-DS
I've fixed code that is suspected to be causing the problem. After removing current tomotopy package, you can install dev version using following command:
pip install --index-url https://test.pypi.org/simple/ tomotopy

I would appreciate it if you try to do the same code with the dev version.

@bab2min
I have tested the dev version and model.infer() worked fine, generated topic probabilities for all 12k documents.
However, convert_to_lda  is causing the kernel to die.

@Jurgita-DS
Uhmmm, could you share the value for docs' uid of Corpus feeded to HDPModel if you provided uid for docs?

@bab2min
How do I get uid of Corpus?
This is how I created corpus:
corpus = tp.utils.Corpus()
for doc in texts:
    corpus.add_doc(words = doc)

@Jurgita-DS
Aha, you created without uid param. I'll check it. Thank you!

Hi, I am using Python 3.8 and Tomotopy 0.11.1, having a similar issue as @Jurgita-DS . I tried to downgrade to tomotopy==0.10.2 and the problem reproduces equally.
While working both on Jupyter various methods would kill my kernel - get_topic_dist and get_topics - in particular. As Jupyter wouldn't provide me with loads of debugging information I run my script from the terminal where I got a segmentation fault. This is not a data dimension issues, as this happen also when i am trying these methods on 1 single document created with make_doc. The infer method seems to work in small samples.
Any news on this? Thank you!

Hi @giuliaok ,
Sorry for providing the unstable version.
Could you share your code that caused a segmentation fault?
I cannot reproduce similar problems in my setting, thus I have a difficulty to figure out that bug.
If you have code or data that cannot provide publicly, it is okay to send them to my personal email (bab2min@gmail.com). I'll use them for only debugging, not any other purpose and delete them after finding the cause.

Hi, thank you for coming back and no apologies needed, I love the package!
I have issues in reproducing the problem myself. If I have a model, let's say:

mdl = tp.HDPModel(tw = tp.TermWeight.IDF, min_cf = 30, rm_top = 20, gamma = 0.8, alpha = 0.1, initial_k = 12, seed = 99999)

and add documents to it with:
for vec in documents:
    mdl.add_doc(vec)

where documents is whichever type of text list, I then train the model as:
mdl.burn_in = 1000
mcmc_iter = 5000
for i in range(0, mcmc_iter, 100):
    mdl.train(100, workers = 3)

Until here everything works perfectly. When I try to apply whichever method to the trained model, such as for instance mdl.summary() the kernel randomly dies due to a segmentation fault. What is even more mysterious to me is that this does not happen all the times but completely at random: for instance, at the moment I have two Jupyter Notebooks opened on the same virtual environment (so same dependencies, same python version etc.) - on one notebook I am able to use all methods, on the other I get a segmentation fault as soon as I try to use any method. I am working on MacOS Catalina.
I understand this sounds very weird but I swear this is what is happening 😂
Thanks!

@giuliaok
Thank you for sharing your detail. I'll examine it.

@giuliaok, could you do me a favor?
Can you check if the kernel randomly dies in your environment when running the same code but without argument min_cf = 30, rm_top = 20 ?

@giuliaok
Honestly, I haven't yet figured out the obvious cause of the problem.
But, I found a bug where words excluded by min_cf or min_df were given an inappropriate topic id and it may cause a crash at showing topic information of those words.
I've updated a new test version. I'd appreciate it if you can run the same with the new version  to see if the problem occurs.
$ pip install -U --index-url https://test.pypi.org/simple/ tomotopy==0.12.0 # install a test version

@bab2min
I can confirm the kernel dies regardless of whether you specify min_cf, rm_top or min_df. To me it happens when working with a PLDAModel which I train on a corpus of ~6,000 documents with each document having at most a character length of 1,000,000. After training the model sucessfully infering the topic distribution of a new document via make_doc() and get_topic_distribution() causes the kernel to die. Strangely enough, and similar to the problem of @Jurgita-DS, the kernel only dies if I infer topic distributions of a list of documents. If I infer the topic distribution of the documents individually (i.e. not via a list comprehension or a loop), everything seems to work fine.
My first hunch is that the problem might be a storage related issue. The trained PLDAModel is quite large (~900 MB) and the list of documents for which I want to infer topic distributions is around 600 MB.
Although I cannot share my data, I am happy to assist in solving the issue. I think your package is a great contibution! So do let me know if you need additional info.
Best
Julian

Hi @julienOlivier3
Thank you for sharing your experience.
Can you test if the crash still occurs when you provide a small list of docs (2 or 3)?
In my opinion, if it's a storage issue, it shouldn't happen when a short list is given, but only when a long list is given.
Otherwise, there may be some bugs in implementation of infer for multiple docs.

@mmawada
Yes, I checked the mail.
I tried to reproduce them but failed. It seems to be a problem that is probably only happening with certain data. Could you share your code piece and data?
If it is difficult for you to share your code or data publicly, it is okay to send them to my email.

The data files are over 600MB , where can I share them with you privately
?
On Sat, 28 Aug 2021 at 00:34, bab2min @.***> wrote:

@mmawada https://github.com/mmawada
Yes, I checked the mail.
I tried to reproduce them but failed. It seems to be a problem that is
probably only happening with certain data. Could you share your code piece
and data?
If it is difficult for you to share your code or data publicly, it is okay
to send them to my email.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bab2min/tomotopy/issues/62#issuecomment-907566381,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIUSPF46OC2F4FUEJ64EI7DT7BRMNANCNFSM4OBFPFQQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

I attach the data files below and the code .
hdp_model_eqwgts.bin
https://drive.google.com/file/d/1rRX0alChLrjCoTIzD9G_472I68xZ2pJv/view?usp=drive_web
hdp_model_pmi.bin
https://drive.google.com/file/d/1JOPdsi4Lh-o3II-Kfu5tc34hUJsVVIDE/view?usp=drive_web
With the same data the convert_to_lda was working with version 10 , but
with the new version 12 , the kernel is dying so it is definitely a code
bug , memory issues etc.
here are the
the codes are :
hdp_eqwgts = tp.HDPModel.load("models/hdp_model_eqwgts.bin")
hdp_pmi = tp.HDPModel.load("models/hdp_model_pmi.bin")
hdp_idf = tp.HDPModel.load("models/hdp_model_idf.bin")
hdp_idf3 = tp.HDPModel.load("models/hdp_model_idf3.bin")
You can load any of the above hdp and will crash the kernel .  for example
you can try :
use the best IDF model for HDP
ldaimplied_idf, new_topic_id =
hdp_idf3.convert_to_lda(topic_threshold=0.015)
or
use the best model given by coherence test which is of HDP PMI save from
the bin
ldaimplied_pmi, new_topic_id = hdp_pmi.convert_to_lda(topic_threshold=0.015)
Look forward to hearing from you.
Thank you,
Mark
On Sat, 28 Aug 2021 at 00:34, bab2min @.***> wrote:

@mmawada https://github.com/mmawada
Yes, I checked the mail.
I tried to reproduce them but failed. It seems to be a problem that is
probably only happening with certain data. Could you share your code piece
and data?
If it is difficult for you to share your code or data publicly, it is okay
to send them to my email.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bab2min/tomotopy/issues/62#issuecomment-907566381,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIUSPF46OC2F4FUEJ64EI7DT7BRMNANCNFSM4OBFPFQQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Thanks @mmawada
I've received the file you shared well. I'll examine that bug!

@mmawada
It was confirmed that the cause of the problem was that the words removed from the modeling by min_cf, min_df or rm_top were not properly processed in the converting to the LDA model.
The problem can be reproduced when calling convert_to_lda of HDP model with min_cf > 0, min_df > 0 or rm_top > 0.
This bug will be patched in the next update. Thank you for reporting the bug again!

I am very glad you found the bug . Could you send me the updated file as
soon as possible, please?  I would I appreciate it if you email it to me
too .
Thank you very much,
Mark
On Tue, 31 Aug 2021 at 13:31, bab2min @.***> wrote:

@mmawada https://github.com/mmawada
It was confirmed that the cause of the problem was that the words removed
from the modeling by min_cf, min_df or rm_top were not properly processed
in the converting to the LDA model.
The problem can be reproduced when calling convert_to_lda of HDP model
with min_cf > 0, min_df > 0 or rm_top > 0.
This bug will be patched in the next update. Thank you for reporting the
bug again!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bab2min/tomotopy/issues/62#issuecomment-909441426,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIUSPF6PLAFQDKI224C4NY3T7UGXJANCNFSM4OBFPFQQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@Jurgita-DS
Aha, you created without uid param. I'll check it. Thank you!

I am having an issue when training an LDA model, I get 'uid' values of  '' for all documents. I also don't see any option to provide document ids to the Corpus as you mention here. Is there the capability of including user defined document ids?

@mmawada
Now a test binary for v0.12.2 is building. After building, you can get it using following command:
$ pip install -U --index-url https://test.pypi.org/simple/ tomotopy==0.12.2

Thanks a lot! Please let me know when the building is done.
On Thu, 2 Sept 2021 at 00:09, bab2min @.***> wrote:

@mmawada https://github.com/mmawada
Now a test binary for v0.12.2 is building. After building, you can get it
using following command:
$ pip install -U --index-url https://test.pypi.org/simple/ tomotopy==0.12.2
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bab2min/tomotopy/issues/62#issuecomment-911140340,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIUSPF6LE4DQNCSCGIELKVDT732JBANCNFSM4OBFPFQQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@mmawada Done! (https://github.com/bab2min/tomotopy/actions/runs/1192523849)

Awesome, thanks!
On Fri, 3 Sept 2021 at 08:31, bab2min @.***> wrote:

@mmawada https://github.com/mmawada Done! (
https://github.com/bab2min/tomotopy/actions/runs/1192523849)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bab2min/tomotopy/issues/62#issuecomment-912502220,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIUSPF3SG2P76ZQLBZQ5S3TUAC52BANCNFSM4OBFPFQQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


can't install tensorflow-addons==0.6.0

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): mac version:10.12.6
TensorFlow version and how it was installed (source or binary): didnt install
TensorFlow-Addons version and how it was installed (source or binary): pip install tensorflow-addons==0.6.0
Python version: python 3.6.9
Is GPU used? (yes/no):no

Describe the bug
ERROR: Could not find a version that satisfies the requirement tensorflow-addons==0.6.0 (from versions: 0.3.1)
ERROR: No matching distribution found for tensorflow-addons==0.6.0
Code to reproduce the issue
i created a conda virtual enviroment and activated it, then run
pip install tensorflow-addons==0.6.0
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Other info / logs
my pip == 19.3.1
numpy == 1.17.4
and when i try "pip install tensorflow-addons",  tensorflow-addons==0.3.1 will be installed which is not what i want.
thanks for help!

and I wanna know what kind of enviroment does tensorflow-addons==0.6.0 need? thanks

I find out that I can't install it because my Mac version is 10.12
update to 10.15 then it can be installed


Settings app logo doesn't appear

Describe the bug
Settings app logo doesn't appear
To Reproduce
Open settings
Expected behavior
It should be the settings app
Screenshots

Desktop (please complete the following information):

Windows 10 LTSC

It happens on all UWP apps, I will try to do something for that


docs: blog post improving retrieval and reranking for RAG

Description
We added the blog post on fine-tuning the sentence similarity model for RAG
Closes #3896
Type of change
(Remember to title the PR according to the type of change)

[X] Documentation update

How Has This Been Tested
(Please describe the tests that you ran to verify your changes.)

[X] sphinx-autobuild (read Developer Documentation for more details)

Checklist

[X] I added relevant documentation
[X] I followed the style guidelines of this project
[X] I did a self-review of my code
[X] I made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I filled out the contributor form (see text above)
[ ] I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

Hi @sdiazlor, looking good already. I have some starting comments.

Would you be able to align your approach with the FeedbackDataset.for_sentence_similarity or FeedbackDataset.for_retrieval_augmented_generation https://github.com/argilla-io/argilla/pull/3833?
The images don't seem to be working.
I think you can go a bit more in-depth into retrieving the data from a vector search index. Something like deepset  can be used.

Hi @sdiazlor, the tutorial is looking great already. How are you going to close the loop of retrieving documents and optimizing / labelling which are more similar based on the rag retrieval efforts? I think we need two datasets: one to assess the similarity of the sentences compared to the query and one to assess the similarity between sentences.

@davidberenstein1957 I made some changes according to what we discussed

Thanks @sdiazlor, you can walk us through them during the daily :)

Issue created for that #4088

Hi @sdiazlor, could you align this with the most recent update from Agus w.r.t. changing the RankingQuestion to the RatingQuestion? Also, I think we need to update the images to align with the actual look of the datasets within the UI.

Hi @sdiazlor, could you align this with the most recent update from Agus w.r.t. changing the RankingQuestion to the RatingQuestion? Also, I think we need to update the images to align with the actual look of the datasets within the UI.

@davidberenstein1957 the ratingquestion has already been updated in the latest commit, the images are what are remaining due to the frontend problem I was having

@sdiazlor, is this ready to merge?

@davidberenstein1957 yes, we can merge it in develop as you mentioned


Add type to all primary and linked resources

Ensure id is expressed as a string in the data

"Each resource object MUST contain a type member, whose value MUST be a string."

https://github.com/dgeb/json-api/blob/v1rc2/format/index.md#resource-ids-

:+1:


NoneType' object has no attribute 'find_all

Describe the bug

i am finding this, it worked yesterday just fine but today
as i was trying to download and still trying the downloader is giving this
message
https://imgur.com/a/ESyMCXE

To reproduce

LOG

I don't think 9anime works.

9anime is down at that time. Now it works.

It's stopped working again. Unable to download from 9anime

Same issue here
Tried all providers
$anime watch --new 'berserk' --provider PROVIDER_NAME
9anime - 'NoneType' object has no attribute 'find_all'
kissanime - random anime matches instead of the search term
kisscartoon - 'NoneType' object has no attribute 'find_all'
masterani -  Expecting value: line 1 column 1 (char 0)
twist.moe - ModuleNotFoundError: No module named 'Crypto'

anime-downloader 3.6.3
anime: 'NoneType' object has no attribute 'find_all

Same issue here, seemingly regardless of what command I use, search term or provider:
$> anime dl 'code geass'
anime: anime-downloader 3.6.3
anime: 'NoneType' object has no attribute 'find_all'
To get into this state, from a new homebrew install of anime-downloader, I ran:
$> anime dl 'code geass'
[valid output, Ctrl-C to exit]
$> anime dl fullmetal
[valid output, entered a number from list]
Enter the anime no:  [1]:
anime: Selected Fullmetal Alchemist (Dub)
anime: Extracting episode info from page
Expecting value: line 1 column 1 (char 0)
From that point on, all commands (except "watch --list") give the same error:
$> anime watch --list
anime: anime-downloader 3.6.3
anime: Add something to watch list first.
$> anime dl fullmetal
anime: anime-downloader 3.6.3
anime: 'NoneType' object has no attribute 'find_all'
$> anime dl 'code geass'
anime: anime-downloader 3.6.3
anime: 'NoneType' object has no attribute 'find_all'
$> anime watch --new 'berserk'
anime: anime-downloader 3.6.3
anime: 'NoneType' object has no attribute 'find_all'


Pivot table name does not use a plural form

Laravel Version
10.48.12
PHP Version
8.1.29
Database Driver & Version
No response
Description
https://laravel.com/docs/10.x/eloquent#table-names says "By convention, the "snake case", plural name of the class will be used as the table name unless another name is explicitly specified. So, in this case, Eloquent will assume the Flight model stores records in the flights table, while an AirTrafficController model would store records in an air_traffic_controllers table"
It doesn't look like so for pivots:
class PackageSku extends \Illuminate\Database\Eloquent\Relations\Pivot
{
}

PackageSku::all();

Results in use of package_sku table, while it should be using package_skus.
Steps To Reproduce
class PackageSku extends \Illuminate\Database\Eloquent\Relations\Pivot
{
}

PackageSku::all();

I just found #30788. So, not a bug, but the documentation at https://laravel.com/docs/10.x/eloquent#table-names should probably mention that to limit confusion. This is unexpected behavior because Pivot extends Model.

Documentation refers to it, on the many-to-many section:

To define this relationship, three database tables are needed: users, roles, and role_user. The role_user table is derived from the alphabetical order of the related model names and contains user_id and role_id columns. This table is used as an intermediate table linking the users and roles.

https://laravel.com/docs/10.x/eloquent-relationships#many-to-many-table-structure

Thanks @rodrigopedra