Issue with Symfony 4.4

My composer.json contains:
"require": {
     "php": "^7.1.3",
     "ext-ctype": "*",
    "ext-iconv": "*",
    "ext-json": "*",
    "api-platform/api-pack": "^1.1",
    "beberlei/doctrineextensions": "^1.1",
    "guzzlehttp/guzzle": "^6.3",
    "nelmio/cors-bundle": "^1.5",
    "sensio/framework-extra-bundle": "^5.2",
    "symfony/asset": "^4.1",
    "symfony/console": "^4.1",
    "symfony/framework-bundle": "^4.1",
    "symfony/orm-pack": "^1.0",
    "symfony/templating": "^4.1",
    "symfony/validator": "^4.1",
    "symfony/yaml": "^4.1",
    "friendsofsymfony/oauth-server-bundle": "^1.6"
},
"require-dev": {
    "phpspec/prophecy": "^1.8",
    "symfony/phpunit-bridge": "^3.4 | ^4.0",
    "squizlabs/php_codesniffer": "3.*",
    "edgedesign/phpqa": "^1.23",
    "sensiolabs/security-checker": "^6.0",
    "phpstan/phpstan": "^0.11.19",
    "vimeo/psalm": "^3.6",
    "jakub-onderka/php-parallel-lint": "^1.0",
    "jakub-onderka/php-console-highlighter": "^0.4.0",
    "psalm/plugin-phpunit": "^0.7.0",
    "matthiasnoback/symfony-dependency-injection-test": "^4.0"
},

When Symfony 4.4 was released, composer update did the following:
Package operations: 0 installs, 14 updates, 11 removals
  - Removing league/container (2.4.1)
  - Removing grasmash/yaml-expander (1.4.0)
  - Removing grasmash/expander (1.0.0)
  - Removing doctrine/doctrine-cache-bundle (1.3.5)
  - Removing dflydev/dot-access-data (v1.1.0)
  - Removing container-interop/container-interop (1.2.0)
  - Removing consolidation/output-formatters (3.5.0)
  - Removing consolidation/log (1.1.1)
  - Removing consolidation/config (1.2.1)
  - Removing consolidation/annotated-command (2.12.0)
  - Removing consolidation/self-update (1.1.5)
  - Updating symfony/translation-contracts (v1.1.7 => v2.0.0): Loading from cache
  - Updating symfony/validator (v4.3.8 => v4.4.0): Loading from cache
  - Updating symfony/twig-bridge (v4.4.0 => v5.0.0): Loading from cache
  - Updating symfony/twig-bundle (v4.3.8 => v4.4.0): Loading from cache
  - Updating symfony/security-csrf (v4.4.0 => v5.0.0): Loading from cache
  - Updating symfony/filesystem (v4.4.0 => v5.0.0): Loading from cache
  - Updating symfony/security-bundle (v4.3.8 => v4.4.0): Loading from cache
  - Downgrading consolidation/robo (1.4.11 => 0.1.1): Loading from cache
  - Downgrading pdepend/pdepend (2.5.2 => 2.2.0): Loading from cache
  - Removing phpmd/phpmd (2.7.0)
  - Installing phpmd/phpmd (2.5.0): Loading from cache
  - Updating symfony/process (v4.4.0 => v5.0.0): Loading from cache
  - Updating symfony/framework-bundle (v4.3.8 => v4.4.0): Loading from cache
  - Updating symfony/doctrine-bridge (v3.4.35 => v4.4.0): Loading from cache
  - Updating doctrine/doctrine-bundle (1.12.1 => 2.0.2): Loading from cache

And following these operations, trying to run PHPQA fails with the following:
PHP Fatal error:  Uncaught Error: Class 'RoboFile' not found in /bundle/vendor/consolidation/robo/src/Runner.php:61
Stack trace:
#0 /bundle/vendor/edgedesign/phpqa/phpqa(55): Robo\Runner->execute(Array)
#1 {main}
  thrown in /bundle/vendor/consolidation/robo/src/Runner.php on line 61

Fatal error: Uncaught Error: Class 'RoboFile' not found in /bundle/vendor/consolidation/robo/src/Runner.php on line 61

Error: Class 'RoboFile' not found in /bundle/vendor/consolidation/robo/src/Runner.php on line 61

Call Stack:
    0.0002     404792   1. {main}() /bundle/vendor/edgedesign/phpqa/phpqa:0
    0.0779    1779304   2. QARunner->execute() /bundle/vendor/edgedesign/phpqa/phpqa:55

I'm guessing the issue is related to the downgrading of consolidation/robo, but I cannot for the life of me work out why this is happening. I don't even know if this is a PHPQA issue, but I'm hoping someone can either tell me what's wrong, or point out how you can find out why a package was downgraded in Composer.
Thanks.
Edit: here's some extra info. By adding "consolidation/robo": "^1", into require-dev and running composer update, the following happens:
Package operations: 10 installs, 5 updates, 0 removals
  - Downgrading symfony/filesystem (v5.0.0 => v4.4.0): Loading from cache
  - Downgrading symfony/process (v5.0.0 => v4.4.0): Loading from cache
  - Installing dflydev/dot-access-data (v1.1.0): Loading from cache
  - Installing grasmash/yaml-expander (1.4.0): Loading from cache
  - Installing consolidation/self-update (1.1.5): Loading from cache
  - Installing consolidation/output-formatters (3.5.0): Loading from cache
  - Installing consolidation/annotated-command (2.12.0): Loading from cache
  - Installing grasmash/expander (1.0.0): Loading from cache
  - Installing consolidation/config (1.2.1): Loading from cache
  - Installing consolidation/log (1.1.1): Loading from cache
  - Installing container-interop/container-interop (1.2.0): Loading from cache
  - Installing league/container (2.4.1): Loading from cache
  - Updating consolidation/robo (0.1.1 => 1.4.11): Loading from cache
  - Updating pdepend/pdepend (2.2.0 => 2.5.2): Loading from cache
  - Removing phpmd/phpmd (2.5.0)
  - Installing phpmd/phpmd (2.7.0): Loading from cache

So it would seem to me that symfony/filesystem and/or symfony/process version 5 is incompatible with some or all of the packages above. Perhaps PHPQA should change the require for consolidation/robo to "consolidation/robo": "~1",, so it is never downgraded?

phpqa won't drop support for older robo. Min version should be restricted, so robo wouldn't be downgraded to v0.1. But that's look like edge-case. You can always restrict robo version in your composer.json.
Related: https://github.com/EdgedesignCZ/phpqa/issues/189#issuecomment-551943785

Yes manually adding the robo version restriction in my composer.json sorted it. Thanks for responding!


Example Application?

Is there a simple example application that could be included in the library?

+1

I also found it hard to get started with library, especially if you're new to Angular. But once you get it, it's working really well. I've created a small application that uses ngDropbox that lets you edit text files in Dropbox here: https://github.com/BangKarlsen/noopad - feel free to use it as a starting point.


Pattern highlight

Description of the new feature/enhancement
iTerm2 has really nice feature - pattern highlighting. You can use complex regexp's to match what you want.
This is very helpful when finding IP addresses, hostnames, etc.

Sorely need this in Windows terminal

So I think broadly we're tracking this in #8849. That one specifically calls out hyperlinks, but I think could be used for any match really. We also briefly mentioned it in #3920. Coulda swore there was something more specific on the reop, but those two seem to cover it.
Thanks!
/dup #8849


order

On the page: http://eudat-b2safe.github.io/B2HANDLE/creatingclientcertificates.html#creatingclientcertificate
it states "Creating the certificate file":
And than it has:

This can be done using openssl without specifying a subject:
This can be done using openssl with specifying a subject:
The tool is then going to prompt for some information if you don not specify a subject.

Would it better if it states:
This can be done in 2 ways:
Case#1

Creating the certificate can be done using openssl with specifying a subject:

Done
Case#2

Creating the certificate can be done using openssl without specifying a subject:
The tool is then going to prompt for some information if you don not specify a subject.

Done
It also states "don not" and "do not" is a little better english.

This has been addressed in PR #107


Containerize your microservice using Docker

As a Developer
I need the containerized project
So that I will be able to deploy and build the project easier.
Details and Assumptions

Make a Dockerfile.
Use alpine versions to keep low size images.

Acceptance Criteria
Given A Dockerfile
When I run the script to build the image and up a container
Then I must get an image and a container running with the service.


InternalError (see above for traceback): Dst tensor is not initialized.

GPU: 2080ti
setup light=false: InternalError (see above for traceback): Dst tensor is not initialized.
setup light=True: ok

what is the Dst tensor ?

same error
might be because memory is not enough


When i include Endpoints.php

Parse error: syntax error, unexpected '.', expecting ',' or ';' /scrapper/src/InstagramScraper/Endpoints.php on line 8

@d3vilsn0w please update to v0.2.1 and check

Just downloaded latest version and getting same error.

What version of PHP are you using? @d3vilsn0w

I will update library to support older than 5.6 versions

PHP Version 5.5.36, Can you please update it now, As i really need to get it working asap. Thanks a lot. @raiym

yes

Thank you, I'm waiting.

It is done. please check

That was fast. I can confirm it is working fine now. Thanks again.

Cool

Thank you!


docs.rs documentation improvement

Category
Documentation and code comments
Describe the feature you'd like to request
On the Rust crate documentation (e.g., https://docs.rs/cedar-policy/latest/cedar_policy/) there are no usage examples on the landing pages, just snippet examples for the particular API calls. When you get to the landing page, you are looking for help, but instead you get a long list of types and API calls with no explanation.
I like the worked example we have on the crates.io page for cedar-policy; could we something like that for our various packages, too? I see this is done for other Rust packages, e.g., https://docs.rs/graph/latest/graph/ and https://docs.rs/bigdecimal/latest/bigdecimal/ as two random cases.
Describe alternatives you've considered
Not really. Doing nothing = status quo.
Additional context
No response
Is this something that you'd be interested in working on?

[ ] 👋 I may be able to implement this feature request
[ ] ⚠ This feature might incur a breaking change

A low-effort improvement to the top level docs could be using #![doc = include_str!("../README.md")] to include the readme verbatim in the crate docs, getting the example we have there for free. The exact text/examples we want in each place may be different though.


Move rhumbBearing implementation to a new PR

@armantorkzaban Did we do it, so we can move forward with the merge of this one?

done #109


ObjectNav performance

Hi Apoorv,
I have rerun the training of Habitat-ObjNav with the default configurations, trying to reproduce your results. I found that for the val-mini set which is evaluated during training, the SR peaks at 150M frames (16.67%), but then it drops to 0%. Does this behavior also appear in your experiments?
I also noticed that you have provided the pre-trained model of objectnav-rgb-clip at the 175M step, does it mean that you use the val-mini or val set to choose the best model? Thanks!

Hi, no, I didn’t observe this behavior during training. The val-mini metrics peaked at 175M steps, but did not drop significantly after this. And, we saw that this checkpoint was best according to both the val-mini and val sets.

It might also be a good idea to run evaluation on the checkpoint I have provided to see if you are able to reproduce the metrics from our paper.

Hi Apoorv, thanks for pointing this out.
A small question: I am testing the evaluation on the val-unseen split, but I found the speed is extremely slow -- eval a single checkpoint could take upto 20+ hours. I know that there are lots of episodes and the max_allowed_steps is large and it takes time for the simulator to renender the environment / move the agent, but I have already set NUM_ENVIRONMENTS=11 (equals to the #val-unseen envs) and use_cuda and disabled generate_videos. Does this also happened on your side? Any suggestions how to speed things up?
Many thanks!

Hi Yicong, glad you were able to resolve that issue! It certainly takes a while to evaluate checkpoints, although my experience was closer to a few hours rather than 20. If you would like to open a PR for the habitat branch with this fix, I will consider merging it.

Hi Apoorve, I got some new results which really confuse me.
I run a very simple baseline which uses CLIP for RGB and PointNav-Resnet for Depth features (this is the only thing I changed in the code), I use the same policy network as in your paper, and the default DDPPO configurations. I trained the exactly same model twice for 250M steps, the two results have 7% discrepency (15% vs 22% success rate) on the val-unseen data.
Do you have similar observation in your experiments? I am wondering if such instability is due to the DDPPO method or the ObjectNav data? From the recent publications on Object Nav, 7% is a huge gap and 22% success rate is already SoTA. With such instability I am really not sure how to more on to further experiment.
Any ideas? Any suggestions how I can make the DDPPO training more stable?
Thanks!

Hi, sorry to miss this earlier. I'm not sure how to make DD-PPO training more stable, however I did see fluctuations in model performance. I recommend that you save checkpoints for every 25M or 50M steps and select the one with the best validation metrics. The checkpoint at 250M steps is not necessarily the best performing model. Hope that helps.


ng serve broken due to a special char in path

Which @angular/* package(s) are the source of the bug?
Don't known / other
Is this a regression?
No
Description
Using the command
ng serve --open
it results in alot of "Module build failed: Cannot find module xxx".
The problem is due by the presence of a special char in the path (eg.: C:\Sources\C# API\AppName\Ui)
"C# API" il the problem, if I named it "C_API" everything work fine.
Please provide a link to a minimal reproduction of the bug
No response
Please provide the exception or error you saw
No response
Please provide the environment you discovered this bug in
Angular CLI: 12.2.2
Node: 14.17.5
Package Manager: npm 7.21.0
OS: win32 x64

Angular: 12.2.2
... animations, cli, common, compiler, compiler-cli, core, forms
... localize, platform-browser, platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1202.2
@angular-devkit/build-angular   12.2.2
@angular-devkit/core            12.2.2
@angular-devkit/schematics      12.2.2
@schematics/angular             12.2.2
rxjs                            6.6.6
typescript                      4.3.5

Anything else?
No response

Same here, # is the problem in the path

I'm sorry, but this issue is not caused by Angular CLI. This is a Webpack issue, please see https://github.com/webpack/webpack/issues/14755


Uncaught (in promise) DOMException

Lobibox version: 1.2.7
Follow Code:
Lobibox.notify('success', {
    sound: true,
    pauseDelayOnHover: true,
    continueDelayOnInactiveTab: false,
    title: 'Title',
    size: 'mini',
    msg: 'message',
    iconSource: "fontAwesome",
    soundPath: '/lib/lobibox/dist/sounds/',
});

Sometimes this mistake happens. The sound rings and does not ring. What he did not touch, he made an exception below:

Uncaught (in promise) DOMException notifications.min.js:1

Hi, Im dealing with same problem. Any progress in this matter?

No, I'm waiting for the solution.

Unfortunately, I could not reproduce this problem.
Can you provide step by step instructions to reproduce it?

seems it's cause of the sound play.

first time is ok, then error occurred if reload


fix: do not start multiple blockchain plugin on retry of namespace start

Make sure that in the case where the NS fails to restart because of another plugin has failed do not start the blockchain plugin again when it was successfully started before.
Played around with the idea of using context to clear everything but it didn't make sense. Also I don't think a mutexLock is needed for this.
If you are wondering how all the other plugins work it's because at initialisation part those plugins return a struct that we then use as part of starting the managers/plugins.  So we can check for the existence of that struct and not re-initialise that manager/plugin. In the case of the blockchain plugin, it's a shared struct and we just start a new event streams in the init part.

Fixes https://github.com/hyperledger/firefly/issues/1527


Add ability to Mauro to generate Audit Log

When completing the new Mauro entry for the Unified Register, it asked about an Audit Log.
I asked James if Mauro had an audit log to show what items have been edited by each user.
James said this exists but may not be in a good state for us. He did say he could look at changing it and add a button if we wanted to generate it.
It is not urgent until after migration.

Cath to advise of the requirements.


[Suggestion] Replace lines 126 - 130 so that the terrain cost is terrain -1, to allow for both difficult terrain and movement penalty

Hello,
I don't know how this would turn out so forgive me if this has higher implications. My suggestion is as follows: instead of return'ing 1 or 2 (for water), I believe a better option would be to subtract the configured terrain cost by one (minimum 1).
This would allow users to increase the terrain cost to reflect difficult terrain independent of the costs associated with the terrain you're on.
For instance, if a the terrain cost associated with a tile were to be 3 (1 for water, 1 for other source of difficult terrain), having swimming speed would make that terrain's cost be 2 (3-1) but 3 for those who do not have swimming speed.
Conversely, if the terrain's cost is 2, then people with swimming speed would be 2-1 = 1 (normal speed) while those without swimming speed it would be 2.

Hey Juan,
That's a great idea and it's close to what I had in mind as part of a bigger rewrite to better handle multiple overlapping terrains. I will add it to my todo list.

I've rewritten how multiple overlapping terrain is handled in v1.3.0. Per Dnd5e rules, difficult terrain does not stack but since water is not technically considered difficult terrain it can stack with difficult terrain. The module now properly stacks these movement costs.
Note that this works slightly different than your initial suggestion, as water terrain with a movement cost of 3 will still be completely ignored by swimming creatures. To achieve what you want please place other terrain on top of the water terrain instead. Swimming creatures will then only ignore the water terrain but the movement cost from all other terrain is still applied (for non-swimming creatures the movement cost will be stacked accordingly).


Add Gmsh

A three-dimensional finite element mesh generator with built-in pre- and post-processing facilities
Homepage
Packaged in other Distros:

Homebrew
AUR (pkgbuild)
Linuxbrew
Fedora
FreeBSD

It seems that this port needs to support Fortran.

We can’t reopen this issue until a PR is opened, unfortunately, consider making a pull request!

@JackBoosY  May I ask where does it say it needs fortran?
https://gitlab.onelab.info/gmsh/gmsh/-/tree/gmsh_4_8_4
It seems this lib can be just build as a standard lib and linked?

Stable now is 4.8.4.

You can create a PR to add this port if possible. @xarthurx

@NancyLi1013
PR here: https://github.com/microsoft/vcpkg/pull/21896
I'm really not good at this and never did one before.
This is what I can do currently, so if there are any issues, especially for build systems and the features, please let me know or you can edit it directly.
Thank you.

Close this issue via #21896 merged.


bilinear initialization of deconv layer

Hi,
I am not quite sure whether the following initialization of deconv layer is a valid implementation of bilinear filter for all input feature channels:
def fill_up_weights(up):
    w = up.weight.data
    f = math.ceil(w.size(2) / 2)
    c = (2 * f - 1 - f % 2) / (2. * f)
    for i in range(w.size(2)):
        for j in range(w.size(3)):
            w[0, 0, i, j] = \
                (1 - math.fabs(i / f - c)) * (1 - math.fabs(j / f - c))
    for c in range(1, w.size(0)):
        w[c, 0, :, :] = w[0, 0, :, :]

It doesn't initialize w in dim 1 except for the first input channel.

the up layer is a grouped convolution with the group number equal to the input channels. So the input channel number for each filter is 1.

Got it, thanks!

Hi,
I've got another question why the bilinear initialized kernel not normalized. The norm of the kernel would be 4 if upsampling factor was 2, 16 if upsampling factor was 4 and 64 if upsampling factor was 8.
I used the testing code below:
print(w[0, 0, :, :].sum())

It will amplify the feature values and change the distribution of feature statistics particularly at the earlier stages of training. Does it design on purpose? Will you observe difference results if you changed the initialization method to kaiming normal in or default init or use bn after up sampling conv?
Thanks for your attention


training&pretraining

Excuse me
I have observed that 60 processes are used when reading data, can I change to 5 processes , because my gpu is only 3060. Does this only affect the convergence time and not the final effect ?
At the same time I have another question, because I train on docker, the screen transfer speed is too slow is it possible to cancel the gui. so can i close the gui screen?
Finally, may I ask which file of the code your clip-vn-navigation-model file is in? I have download “objectnav-robothor-clip-rn50.130M.pt” I printed the weights of this model and looked at the size of the model, but I found the size of the model and the name of each layer of the network, and found that this is not the same as the clip model

Hey, sorry for the delayed response. You should be able to reduce the number of processes, but this will increase training time. Converging at the same results may take longer than 30x (= 60 / 5) original training time.
I don't think you can close the GUI screen, but you can run AI2-THOR in headless mode, so screens won't appear. Please see more details at AllenAct Distributed ObjectNav Tutorial.
I am not sure what you are referring to with the particular keyword clip-vn-navigation-model. The weights in objectnav-robothor-clip-rn50.130M.pt don't correspond to CLIP's image encoder network, but the whole embodied agent pipeline (including the visual encoder, but also the RNN and actor/critic networks). Please refer to Figure 2 of our paper.


Cloak Spell Deletes Armour if You Die While Cloaked

Hey there, on the 1.16.1 PaperMC server I'm playing on I noticed that when you use the Cloak spell it removes all your armour in order to make you invisible, as it should, normally when the spell ends your armour is properly added back again, but if you die while your armour is removed due to the Cloak spell the armour doesn't drop, and recasting the spell doesn't add it back again.

Thank you for this bug report, that was a good catch. I apologize that it took me so long to fix such a bad bug, but the current release should fix it.


powermetrics: unrecognized sampler: bandwidth

I try to run asitop and get
ASITOP - Performance monitoring CLI tool for Apple Silicon
You can update ASITOP by running pip install asitop --upgrade
Get help at https://github.com/tlkh/asitop
P.S. You are recommended to run ASITOP with sudo asitop
[1/3] Loading ASITOP
[2/3] Starting powermetrics process
[3/3] Waiting for first reading...
powermetrics: unrecognized sampler: bandwidth
I'm on an M1 Pro 14" MacBook Pro

still experiencing this issue


ImportError : No module named 'enchant'

Hi,
Just moved from django-tinymce to django-tinymce4-lite because I need version TinyMCR V4. I'm seeing this error when loading a page in Django Admin. It looks like this tool depends on enchant, but doesn't explicitly list in as a dependency (hence the bug).
Full Stack Trace:
File "/usr/local/lib/python3.4/site-packages/django/core/handlers/exception.py" in inner 41.             response = get_response(request)
File "/usr/local/lib/python3.4/site-packages/django/core/handlers/base.py" in _legacy_get_response 249.             response = self._get_response(request)
File "/usr/local/lib/python3.4/site-packages/django/core/handlers/base.py" in _get_response 187.                 response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.4/site-packages/django/core/handlers/base.py" in _get_response 185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in wrapper 551.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "/usr/local/lib/python3.4/site-packages/django/utils/decorators.py" in _wrapped_view 149.                     response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.4/site-packages/django/views/decorators/cache.py" in _wrapped_view_func 57.         response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/sites.py" in inner 224.             return view(request, *args, **kwargs)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in change_view 1511.         return self.changeform_view(request, object_id, form_url, extra_context)
File "/usr/local/lib/python3.4/site-packages/django/utils/decorators.py" in _wrapper 67.             return bound_func(*args, **kwargs)
File "/usr/local/lib/python3.4/site-packages/django/utils/decorators.py" in _wrapped_view 149.                     response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.4/site-packages/django/utils/decorators.py" in bound_func 63.                 return func.__get__(self, type(self))(*args2, **kwargs2)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in changeform_view 1408.             return self._changeform_view(request, object_id, form_url, extra_context)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in _changeform_view 1437.         ModelForm = self.get_form(request, obj)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in get_form 615.             fields = flatten_fieldsets(self.get_fieldsets(request, obj))
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in get_fieldsets 304.         return [(None, {'fields': self.get_fields(request, obj)})]
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in get_fields 604.         form = self.get_form(request, obj, fields=None)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in get_form 647.             return modelform_factory(self.model, **defaults)
File "/usr/local/lib/python3.4/site-packages/django/forms/models.py" in modelform_factory 546.     return type(form)(class_name, (form,), form_class_attrs)
File "/usr/local/lib/python3.4/site-packages/django/forms/models.py" in __new__ 252.                                       opts.field_classes)
File "/usr/local/lib/python3.4/site-packages/django/forms/models.py" in fields_for_model 170.             formfield = formfield_callback(f, **kwargs)
File "/usr/local/lib/python3.4/site-packages/django/contrib/admin/options.py" in formfield_for_dbfield 178.                 return db_field.formfield(**kwargs)
File "/usr/local/lib/python3.4/site-packages/tinymce/models.py" in formfield 33.             'widget': TinyMCE(profile=self.tinymce_profile)
File "/usr/local/lib/python3.4/site-packages/tinymce/widgets.py" in __init__ 148.         self.profile = get_language_config()
File "/usr/local/lib/python3.4/site-packages/tinymce/widgets.py" in get_language_config 69.         from enchant import list_languages

Looks like the dependency was removed from the requirements.txt, but the software still depends on it here

I was able to temporarily get around this issue by setting the following in my Django settings:
INYMCE_SPELLCHECKER = False

I cannot reproduce your error. TINYMCE_SPELLCHECKER option defaults to False, that is, without this option in project's settings.py enchant module won't be imported. That is why it is not listed as a dependency - the package does not depend on it by default.

You have described the bug. The software currently supports a flag called TINYMCE_SPELLCHECKER , but it has an implicit dependency on a third party library. The implicit dependency is the bug itself.
Django Tinymce also has this issue, but they handle the exception with a helpful error.
I'm happy to submit a PR to fix. Two options off the top of my head:

Add the dependency to requirements.txt and update docs (best option).
Do what Django Tinymce does and catch this exception (better that current).

Context:

https://12factor.net/dependencies
https://www.python.org/dev/peps/pep-0020/#id3

There is no bug. By default the spellchecker is disabled and you won't get any error. The documentation explicitly states that you need to install pyenchant for the spellchecker to work. If a user does not reads the documentation, it is their problem.
The dependency on pyenchant was removed to address the situation when a user does not use spellchecker or when pyencnant/enchant libraries are not available/broken on their system, like in #1.

Thanks

Although I agree that more explicit error message will be helpful. I will add it in the next release.


Unable to install mama

git clone git@github.com:JonJala/mama.git
Cloning into 'mama'...
<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

That command and repo information looks right at first glance.  Are you
able to clone other public repositories on GitHub?
On Tue, Sep 12, 2023 at 11:40 AM Mahan @.***> wrote:

git clone @.:JonJala/mama.git
Cloning into 'mama'...
@.: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
—
Reply to this email directly, view it on GitHub
https://github.com/JonJala/mama/issues/33, or unsubscribe
https://github.com/notifications/unsubscribe-auth/APIOF5526XFFDVSAK5YXPWDX2B66RANCNFSM6AAAAAA4VBQFGE
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks! I was able to install it:)


Feature Request: gobuster dir，Before detection, make sure that the random directory and file do not exist

Feature Request: gobuster dir
Before detection, make sure that the random directory and file do not exist, ensure the credibility of the status code, and then start detection
1、random path,file name
234lknh2k3hj4/23424s.jsp
now get status code is 404,not 200
2、now start check other cgi
if is 200, so it is exist

Wildcard detection already does this behind the scenes using GUIDs.

That's already done here: https://github.com/OJ/gobuster/blob/master/gobusterdir/gobusterdir.go#L87
If a non existent file return the same output as the root site you will get the wildcard warning


Fixes codemirror error when rendering with angular universal

See https://github.com/codemirror/CodeMirror/issues/3701 - it looks like Codemirror is not going to fix the core lib so that it works with universal apps, so this is a workaround to prevent codemirror being bundled in a way that it breaks universal

Wow, thanks for looking into this.


Doesn't work in Chrome extensions

I tried to load https://news.ycombinator.com/ into a simple extensions popup, but got this error:

You need the other party to grant the operation permission for the access
to work
Vào 04:14, T.4, 15 Th2, 2023 T. H. Doan @.***> đã viết:

I tried to load https://news.ycombinator.com/ into a simple extensions
popup, but got this error:
[image: image]
https://user-images.githubusercontent.com/4110567/218864129-98a7c9c9-06fd-404d-9967-4ab21d47dfac.png
[image: image]
https://user-images.githubusercontent.com/4110567/218864335-265e09e5-a2d4-4a45-ac7c-92dc71d3bcf2.png
[image: image]
https://user-images.githubusercontent.com/4110567/218864407-3eb8bc79-9eb5-43fe-9ded-6b631547fb9e.png
[image: image]
https://user-images.githubusercontent.com/4110567/218864493-c371ab19-4e5c-4db5-85d7-ae168e042f44.png
[image: image]
https://user-images.githubusercontent.com/4110567/218864556-0506021f-e933-4e7c-9b9b-354036ee57eb.png
—
Reply to this email directly, view it on GitHub
https://github.com/niutech/x-frame-bypass/issues/51, or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5FSPE6RZGCIPS6XKXTJ65LWXPYU3ANCNFSM6AAAAAAU4B2CXM
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

@Boot-Now, I do not have control over hackernews. I thought we could use this package to "bypass the X-Frame-Options: deny/sameorigin response header".

You from Vietnamese or where ?
Vào 06:47, T.4, 15 Th2, 2023 T. H. Doan @.***> đã viết:

@Boot-Now https://github.com/Boot-Now, I do not have control over
hackernews. I thought we could use this package to "bypass the
X-Frame-Options: deny/sameorigin response header".
—
Reply to this email directly, view it on GitHub
https://github.com/niutech/x-frame-bypass/issues/51#issuecomment-1430539314,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5FSPE2JPYBAWOYY2QHUDBDWXQKPTANCNFSM6AAAAAAU4B2CXM
.
You are receiving this because you were mentioned.Message ID:
@.***>

You can create your own proxy on Cloudflare by setting up a worker and using the source below.
https://github.com/Zibri/cloudflare-cors-anywhere

Wow, you great !
Vào 06:51, T.4, 15 Th2, 2023 Shane @.***> đã viết:

You can create your own proxy on Cloudflare by setting up a worker and
using the source below.
https://github.com/Zibri/cloudflare-cors-anywhere
—
Reply to this email directly, view it on GitHub
https://github.com/niutech/x-frame-bypass/issues/51#issuecomment-1430541704,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5FSPE6ZALFYGY3KRWN46HTWXQK67ANCNFSM6AAAAAAU4B2CXM
.
You are receiving this because you were mentioned.Message ID:
@.***>

@ShaneOss Let me try to set one up now -- I've never done this before.

Ok, if you pecfect .
Vào 06:53, T.4, 15 Th2, 2023 T. H. Doan @.***> đã viết:

@ShaneOss https://github.com/ShaneOss Let me try to set one up now --
I've never done this before.
—
Reply to this email directly, view it on GitHub
https://github.com/niutech/x-frame-bypass/issues/51#issuecomment-1430544576,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5FSPE5QC345BDWEJQPPXRLWXQLJFANCNFSM6AAAAAAU4B2CXM
.
You are receiving this because you were mentioned.Message ID:
@.***>

@ShaneOss I just created my cloudflare worker and updated x-frame-bypass.js  to point to my own worker, but the iframe is still not loading hackernews. Here's the error:

The only thing I changed in the worker code is this:
whitelist = ["https://www.hackernews.com/"]; // regexp for whitelisted origins

Okay, I made some progress. After adding host_permissions required by extensions, I'm now getting this error:

Is there some code I need to change in x-frame-bypass.js to make it work with the worker code at https://github.com/Zibri/cloudflare-cors-anywhere ?

Debugging the issue in the cloudflare worker console, it seems there's an error in the https://github.com/Zibri/cloudflare-cors-anywhere code:

@ShaneOss I just created my cloudflare worker and updated x-frame-bypass.js  to point to my own worker, but the iframe is still not loading hackernews. Here's the error:

The only thing I changed in the worker code is this:
whitelist = ["https://www.hackernews.com/"]; // regexp for whitelisted origins

My understanding is the whitelist is to only allow those domains to use your proxy. I.e. the domain you're calling from. Are you writing a plugin for www.hackersnews.com, and would be calling it from that domain? If not, I wouldn't specify one.

Hey guys, I managed to solve all issues by using this worker instead :D - https://gist.github.com/Kahtaf/e20515800054efcfb22830b4bc883880
Thanks for your assist.


Please provide a full eco-to-3 letter translation in tab-delimited format

What MGD needs is a simple tab-delimited file of ECO/3-letter equivalent that provides all of the ECO/3-letter equivalents, not just the top nodes.
See that attached file.
[eco.txt](https://github.com/evidenceontology/evidenceontology/files/508254/eco.txt
I am generating this file by reading the eco.obo file, the xrefs, the synonyms.  It would be great if this could be provided by the Evidence Ontology group and we could just pick up this tab-delimited file.  We don't need the entire Ontology.  We just need this piece.

#109
Hi,
We will work on it and add the file to our repository soon.
Thanks for providing the details, that helps.
Thanks,
Suvvi

thank you.

@loricorbani Was this file generated by a script or compiled manually?

by a script that I wrote:
https://github.com/mgijax/goload/blob/master/lib/ecolib.py

@loricorbani
Here's a tab-delimited file that was requested. There is some overlap, especially with EXP, since the classes mapped to some (IDA, for example) are subclasses of the term mapped to EXP. Please let me know if this works for you, or if you would prefer a filtered list with no redundancy.
File Here

thanks.  I have downloaded the file, and will take a look.

@loricorbani Here is a list filtered for redundancy as well, just in case.
File Here

I think that the we need to know which ECO is the "master" or "default" to a given 3-letter code.
That is, "IPI" has 26 ECO ids...but only ECO:0000353 is the "default".
I cannot tell this from the new file.  Can you add a default field to each row?  If "default", then this will be the "master"/top-level field for that 3-letter code?
We need to be able to swing in both directions...ECO-> 3 letter, but also 3-letter -> ECO.
Thanks.

I added in the defaults in the filtered list. Let me know if this works, alternatively I could list the defaults at the top.
File Here

Can you make the "Default" the 3rd field, not the 2nd field?
field 1 : ECO
field 2 : 3 letter code
field 3 : Default or null
Thanks.

Also, we do not need to the "GO_REFS" in this file.
Thanks.

@loricorbani No problem. Removed GO_REFS and put Default in 3rd column here.

Rebecca,
This looks great!  I will change my code to use this new file.  Thank you so much.  This is so helpful.
Lori

I am going to have David Hill take a look as well.  Thanks again!

Everyone should use this: http://purl.obolibrary.org/obo/eco/gaf-eco-mapping.txt
ECO editors - can we make a more prominent link to this on the front page
If there is desire to have a pre-inferred version of this, this is fine but it should follow the same format, be in the same folder in github, and have a similar PURL

Chris,
This file is insufficient for our needs.  We had to write a specific obo parser so we could detect all relationships.
We need to translate:
ECO -> 3 letter code
3 letter code -> ECO default
I requested and am testing a new tab-delimited file from Evidence Ontology (Rebecca…many thanks):
https://github.com/rctauber/ecogo-list/blob/master/eco-go-filter.txt
This file is what MGD needed.  Hopefully it will help others too.
Thanks.
Lori
From: Chris Mungall<EMAIL_ADDRESS>Sent: Wednesday, October 05, 2016 11:54 AM
To: evidenceontology/evidenceontology
Cc: Lori Corbani; Mention
Subject: Re: [evidenceontology/evidenceontology] Please provide a full eco-to-3 letter translation in tab-delimited format (#111)
Everyone should use this: http://purl.obolibrary.org/obo/eco/gaf-eco-mapping.txt
ECO editors - can we make a more prominent link to this on the front page
If there is desire to have a pre-inferred version of this, this is fine but it should follow the same format, be in the same folder in github, and have a similar PURL
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/evidenceontology/evidenceontology/issues/111#issuecomment-251716439, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANhwE_DiPaaqvOrMFBQ62B8HJBFt-XRoks5qw8gGgaJpZM4KNjRk.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

Thank you for the comment, Chris. @loricorbani were made aware of the gaf-eco-mapping.txt file previously. That is the file that all users should be using. But they wanted something different, for whatever reason, so we helped them. They explicitly requested a file that followed a different format. I do not think the file that they requested would be of broad use, and I think that having it in the repo next to the original gaf would only serve to confuse users. So we created it to help them with a specific application. That said, yes, it should have been placed in ECO repo and just labeled with a bunch of warnings, in my opinion. (I might still create a subdir for such types of custom files.)  I haven't had time to finish my reply to @loricorbani explaining the gaf file - there may be some confusion about the use of "default" and GO refs - but I am sending this reply first, before this thread gets too off track.

Marcus,
The reason why we asked for this file is because we are loading GO_Noctua/GPAD data into MGD and need to translate any ECO to its proper 3 letter code.
Then, we need to go back the other way…and generate both a GPAD and a GAF file…to send back to GO.
GPAD requires ECO.  GAF requires 3 letter code.  So, we need to translate things in both directions in our pipeline.
ECO -> 3 letter code
3 letter code -> default ECO
The new file really helps us because it eliminates the need for us to write code to take the obo file and generate this ourselves.  And since you are the keepers of the Evidence Ontology, it seems to make sense to ask you to provide this translation.
I expect that other groups that are loading GO_Noctua/GPAD may also find this useful.
We really appreciate your efforts in supplying this file and that it will be updated along with any changes to the Evidence Ontology.
Our GO_Noctua/GPAD loading will be run weekly (if not daily), in order to help facilitate our GO Curators for using GO_Noctua.
Many thanks.
Lori
From: Marcus Chibucos<EMAIL_ADDRESS>Sent: Wednesday, October 05, 2016 12:32 PM
To: evidenceontology/evidenceontology
Cc: Lori Corbani; Mention
Subject: Re: [evidenceontology/evidenceontology] Please provide a full eco-to-3 letter translation in tab-delimited format (#111)
Thank you for the comment, Chris. @loricorbanihttps://github.com/loricorbani were made aware of the gaf-eco-mapping.txt file previously. That is the file that all users should be using. But they wanted something different, for whatever reason, so we helped them. They explicitly requested a file that followed a different format. I do not think the file that they requested would be of broad use, and I think that having it in the repo next to the original gaf would only serve to confuse users. So we created it to help them with a specific application. That said, yes, it should have been placed in ECO repo and just labeled with a bunch of warnings, in my opinion. (I might still create a subdir for such types of custom files.) I haven't had time to finish my reply to @loricorbanihttps://github.com/loricorbani explaining the gaf file - there may be some confusion about the use of "default" and GO refs - but I am sending this reply first, before this thread gets too off track.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/evidenceontology/evidenceontology/issues/111#issuecomment-251727407, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANhwE7zoiiOwkm0diw5I9B_oAWpU7MG2ks5qw9EZgaJpZM4KNjRk.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

I believe the GO GPAD->GAF converter does this for you. cc @tonysawfordebi

Also everyone be sure to read the docs at the top of http://purl.obolibrary.org/obo/eco/gaf-eco-mapping.txt
This actually maps tuples of (code,ref) to ECO classes. This doesn't generally affect Noctua annotation now (the ref code is generally blank for experimental annotations), but it may in future
# Permanent URL of this document:
#  http://purl.obolibrary.org/obo/eco/gaf-eco-mapping.txt
#
# Always use this URL
#
# About:
# This mapping translates evidence codes used in a GAF to a specific evidence type in ECO
#
# Each mapping is from a pair:
#  1. GAF evidence code (e.g. IEA)
#  2. A GO_REF (found in the GAF Reference column)
# to a class identifier in ECO (http://purl.obolibrary.org/obo/eco.owl)
#  3. ECO ID
#
# If the value 'Default' is in column 2 then this mapping is used if
# no more specific mapping can be found
#
# This document consists of these comments (after #s) followed by
# a set of mappings, each represented as the above 3 columns separated by tabs
#
# For example an IEA supported by
#  http://www.geneontology.org/cgi-bin/references.cgi#GO_REF:0000002
# maps to ECO:0000256 'match to sequence model evidence used in automatic assertion'
#
# To request changes in the mappings, submit an item to
#  https://code.google.com/p/evidenceontology/issues/list
#

(oops, looks like URL of tracker needs changed)

We (GOA) also have the need to be able to map ECO codes to their equivalent GO evidence code, so I created a script to do that based on the structure of the ECO graph and the GOECO xrefs.
I'm not convinced that the mappings in the new file are 100% correct, because it contains these entries:
ECO:0001828	(visual sequence inspection evidence) => IC
ECO:0005611	(inference from experimental data evidence) => IDA
ECO:0005612	(inference from phenotype manipulation evidence) => IPI
ECO:0005613	(inference by association of genotype from phenotype) => EXP
These are raw evidence types, and are thus not directly usable in GO annotations. Indeed, it appears that the "... used in manual assertion" forms of these terms do not currently exist, and so should probably be added to the ontology.

Thanks @tonysawfordebi for these comments. These terms are definitely problematic. We need to address this asap, starting with revisiting #85 .

@loricorbani @tonysawfordebi -
Please see #116. I will be adding gaf-eco-derived.txt to the repo today.

@lreiser: this is the ticket I was talking about

Rebecca, file is working even with last ^M still in the file.  Thanks.


Re-localize text through pipeline

Need to take out the current set of localization files and have them go through our localization system

Is #2199 related?

I think launcher and Image Resizer has items that are localized already but since we didn't loc them, we  need to remove them and get them reprocessed

closing against #5779


fix: up limits for raw

:tada: This PR is included in version 16.8.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add more custom _transforms

String

padStart
padEnd
toNumber
toBoolean
charAtIndex
reverse
slugify (encoded URL format)
concat

Array

push
pop
join
slice
range
sort
reverse
removeDuplicates
max
min
mean
sum
toString

Number

abs
ceil
floor
round
random
toString

Object

keys
values
entries
has
getOrDefault
delete
toString
omit

Misc

type

The example1 in https://jsonnet.org is pretty interesting, maybe we can pull that off by assigning a subTransform Block and reuse it, but instead of for loop, in an assign method with an object with properties and values as input.

Implemented most of them in https://github.com/yakshavingdevs/datadance/pull/42
The leftovers: :smiling_face_with_tear:
Number

abs
ceil
floor
round
random
toString (We can achieve this using number + '')

String

toNumber (Achieved through parseInt and parseFloat)
charAtIndex (Achieved through string[idx])
slugify (encoded URL format)
concat (Achieved through str1 + str2)

Array

mean
sum
toString (Achieved through [] | stringify)

Object

getOrDefault (I guess it's unnecessary)
toString (Achieved through {} | stringify)
omit (Achieved through delete)

For various usecases, we would definetly need map, reduce, filter to filter our for each output.

The leftovers are implemented in #43
The example for jsonnet can be built using forEach and sub-transform block.
Similarly, mean, and sum can be built using forEach.
Closing this issue, map, reduce, filter needs a separate issue and discussion on its own.


blobInfo.toBuilder().build() is not equal to blobInfo

We have this situation:
BlobInfo blobInfo = BlobInfo.builder(BlobId.of("b", "n")).build();
assertEquals(blobInfo, blobInfo.toBuilder().build()); // This fails!

This is due to the fact that toBuilder uses builder's setters that initialize (some of them) fields to Data.nullOf() if null is passed. This can be fixed by checking for null before calling setters in toBuilder().

Good catch. How about adding to BlobInfo.Builder a constructor that gets BlobInfo and copy the fields value directly? We could also add a static builder method that accepts BlobInfo...

Adding a constructor to the builder that takes BlobInfo makes sense to me.

Have you considered using AutoValue here? It seems as if it would simplify BlobInfo greatly. It would also have a much more performant equals

I think using AutoValue is a fine idea that we should considered but tracked and dealt with separately.
Created #418


Clear received header values on redirect

Fixes https://github.com/newsboat/newsboat/issues/2230 (at least part of it, I can't verify the roscosmos issues myself).

Coverage decreased (-0.003%) to 59.571% when pulling 0db7c2ea8a8a3afb9d2b8002a77b2a8ff61e9ed8 on dennisschagt:bugfix/issue-2230 into 76e00a2e1b71aece69046298b91827e6bd0d5586 on newsboat:master.


Database error when rescanning

I tried to rescan at a higher with with the command:
monero-lws-admin rescan 2275000 423hS3xz...ECBEGAs
then I got an error in the logs:

2022-06-08 14:56:12.791	I Processed 999 block(s) against 2 account(s)
2022-06-08 14:56:15.777	I Processed 999 block(s) against 2 account(s)
2022-06-08 14:56:19.026	I Processed 999 block(s) against 2 account(s)
2022-06-08 14:56:21.437	I Processed 999 block(s) against 2 account(s)
2022-06-08 14:56:23.968	E Failed to update accounts on disk (thrown at scanner.cpp:496): MDB_NOTFOUND: No matching key/data pair found
2022-06-08 14:56:23.973	D Problems at ssl shutdown: uninitialized
2022-06-08 14:56:23.974	D Problems at cancel: Bad file descriptor
2022-06-08 14:56:23.974	D Problems at shutdown: Bad file descriptor
2022-06-08 14:56:23.987	T [sock -1] Socket destroyed without shutdown.
2022-06-08 14:56:23.988	T [sock -1] Socket destroyed
2022-06-08 14:56:23.988	D Destructing connection #27 to <IP_ADDRESS>
2022-06-08 14:56:23.988	T [sock 44] Socket destroyed without shutdown.
2022-06-08 14:56:23.988	T Closed connection from host <IP_ADDRESS>: 8
2022-06-08 14:56:23.988	T [sock 44] Socket destroyed

Now the container keeps restarting but looks like it can't load the DB again:
root@mainnet:/opt/xmr.sh# docker logs monero-lws --tail 20 -f
2022-06-08 15:03:08.724	I Starting blockchain sync with daemon
2022-06-08 15:03:08.731	I [PARSE URI] regex not matched for uri: ^((.*?)://)?(\[(.*)\](:(\d+))?)(.*)?
2022-06-08 15:03:08.731	I Binding on <IP_ADDRESS> (IPv4):8000
2022-06-08 15:03:08.731	D start accept (IPv4)
2022-06-08 15:03:08.731	D Spawned connection #0 to <IP_ADDRESS> currently we have sockets count:1
2022-06-08 15:03:08.731	D test, connection constructor set m_connection_type=1
2022-06-08 15:03:08.731	I Run net_service loop( 1 threads)...
2022-06-08 15:03:08.732	D Run server thread name: NET
2022-06-08 15:03:08.732	D Reiniting OK.
2022-06-08 15:03:08.732	I Listening for REST clients at http://<IP_ADDRESS>:8000
2022-06-08 15:03:08.732	I Retrieving current active account list
2022-06-08 15:03:08.732	I Starting scan loops on 2 thread(s) with 4 account(s)
2022-06-08 15:03:08.772	I Processed 2 block(s) against 2 account(s)
2022-06-08 15:03:11.705	E Failed to update accounts on disk (thrown at scanner.cpp:496): MDB_NOTFOUND: No matching key/data pair found
2022-06-08 15:03:11.708	D Problems at ssl shutdown: uninitialized
2022-06-08 15:03:11.708	D Problems at cancel: Bad file descriptor
2022-06-08 15:03:11.708	D Problems at shutdown: Bad file descriptor
2022-06-08 15:03:11.709	T [sock -1] Socket destroyed without shutdown.
2022-06-08 15:03:11.709	T [sock -1] Socket destroyed
2022-06-08 15:03:11.709	D Destructing connection #0 to 0.

Yup, you probably have to try again with a lower height to fix the DB now. Good catch, will fix.


Compact filters are downloaded on every restart for an opened channel

Background
I am having an LND node with one channel opened and I saw that on every restart of the node the compact filters are downloaded from the block that contains the funding transaction.

In LND there is a component called ChainArbitrator that creates a ChainWatcher for every active channel in order to be notified with on chain events related to this channel.
As part of its duties, the ChainWatcher registers for a spend notification by using the notifier interface (notifier.RegisterSpendNtfn) to get a notification when the funding transaction output is spent.
When the notifier is configured to be neutrino, it just triggers a rescan starting from the block of the funding transaction, which is implemented by downloading all the compact filters. This seems a correct behavior for the first time but shouldn't be necessary for next times we create a ChainWatcher for this channel. I expect it will start the rescan from the point it stopped.
I have noticed that neutrino has the option to save the filters it downloads, I tried that and it solves this issue but I don't think this is the right solution as we only need to know the "last known scanned" filter for every channel.
Your environment

LND master
macOS
btcd version 0.12.0-beta

Steps to reproduce

Run a simple node configured to work with neutrino and open a channel
Wait for some blocks to be mined or in simnet just generate 1000 blocks to see the issue clearly.
On every restart the filters are downloaded since the funding transaction block even though they were downloaded and the utxo was validated on a previous run.

Expected behaviour
I expect that for every channel the rescan will start from the last point it stopped.
Actual behaviour
The rescan starts from the block contains the funding transaction regardless of where it stopped on a previous run.

See #1788 and the cache in general.

Target is 0.5.1, check out the milestone we're currently using to track the release: https://github.com/lightningnetwork/lnd/milestone/8


maas_local check ansible_fqdn is an issue

Some checks fail in maas_local due to playbook changes :
nova_compute check for instance:
9.0.1
check_details: file=nova_service_check.py,args=--host,args={{ ansible_hostname }},args={{ internal_vip_address }}
9.0.5
check_details: file=nova_service_check.py,args=--host,args={{ ansible_fqdn }},args={{ internal_vip_address }}
This change caused all checks to go fatal because the nova service name is installed using the short hostname, the actual maas check is looking for a FQDN name. So we either revert this change or change the maas checks to look for short hostnames first and then the FQDN hostname.

We are also tracking this here: https://trello.com/c/SwJqrVzY/460-rpc-extras-maas-local-check-ansible-fqdn-is-an-issue

@BjoernT let me see if I understand. This isn't a problem introduced here in rpc-extras, same issue exists in the 9.0.5 version of the integrated maas plays?

Yes it does exist in older releases too.
But I can't file it as os-openstack-ansible since it was generalized, the Rackspace specifics got removed.

Yes, that's fine, just wanted to make sure.

It looks like the check for neutron_linuxbridge_agent_check is also affected by this.

@mattt416 so it looks like you made the change from ansible_hostname to ansible_fqdn. Here is the commit: https://github.com/stackforge/os-ansible-deployment/commit/d7216eccb6e7733f1079e09d99b04ca58634da19.
Unfortunately I can't find issue #452, that probably refers to the old ansible-lxc-rpc repo, and the issues there were removed for some reason.
I can't find any information regarding the problem that made you change this, I really don't understand why ansible_hostname was not giving you the correct thing in the first place.
I can revert these back to ansible_hostname, but what am I breaking if I do that?

@miguelgrinberg Correct -- I appear to have broken this.  :)  The original issue was that we were using ansible_hostname everywhere, which was problematic if someone had deployed nodes w/ a proper FQDN.  The change made was fine except for nova_compute_check and neutron_linuxbridge_agent_check which use the FQDN:
root@aio1_utility_container-0cdd7735:~# nova service-list
+----+------------------+--------------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                                       | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+--------------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 1  | nova-cert        | aio1_nova_cert_container-00a2f79d          | internal | enabled | up    | 2015-03-04T10:25:11.000000 | -               |
| 7  | nova-conductor   | aio1_nova_conductor_container-4a2b2c2d     | internal | enabled | up    | 2015-03-04T10:25:09.000000 | -               |
| 10 | nova-scheduler   | aio1_nova_scheduler_container-c5f35b14     | internal | enabled | up    | 2015-03-04T10:25:07.000000 | -               |
| 13 | nova-consoleauth | aio1_nova_spice_console_container-04b4da91 | internal | enabled | up    | 2015-03-04T10:25:12.000000 | -               |
| 16 | nova-compute     | aio1.rpc.test.com                        | nova     | enabled | up    | 2015-03-04T10:25:05.000000 | -               |
+----+------------------+--------------------------------------------+----------+---------+-------+----------------------------+-----------------+
root@aio1_utility_container-0cdd7735:~#

root@aio1_utility_container-0cdd7735:~# neutron agent-list
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host                                   | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+---------------------------+
| 21cf74a9-62ae-46f9-9135-815c55559eba | Linux bridge agent | aio1.rpc.test.com                    | :-)   | True           | neutron-linuxbridge-agent |
| 2579ffdf-8446-4e56-9ad4-f21973690b0a | Metering agent     | aio1_neutron_agents_container-ea8c3a07 | :-)   | True           | neutron-metering-agent    |
| 29392754-27a9-4f95-84da-695f84b345b4 | DHCP agent         | aio1_neutron_agents_container-ea8c3a07 | :-)   | True           | neutron-dhcp-agent        |
| 49fc7608-dedf-4a53-ab42-ff78badd5fca | L3 agent           | aio1_neutron_agents_container-ea8c3a07 | :-)   | True           | neutron-l3-agent          |
| e9cd0c90-8759-40ef-926c-f054c8cc5ef0 | Metadata agent     | aio1_neutron_agents_container-ea8c3a07 | :-)   | True           | neutron-metadata-agent    |
| f5f45e42-3b47-4f49-a217-f8108d83c8e6 | Linux bridge agent | aio1_neutron_agents_container-ea8c3a07 | :-)   | True           | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+---------------------------+
root@aio1_utility_container-0cdd7735:~#

@mattt416 has kindly volunteered to remove the offending ansible_fqdn calls and test.

I think I better understand this issue now.  As @BjoernT said, we were originally using {{ ansible_hostname }} (short name).  We then received an issue from QE stating that this should be {{ {{ ansible_fqdn }} (FQDN).  At the time, I booted an AIO cloud server w/ an FQDN hostname, verified what they claimed, and made the change.
Having looked at this again, I noticed that a freshly built cloud server AIO w/ FQDN hostname had FQDNs listed for those non-containerised hosts listed in a nova service-list and neutron agent-list.  Chatting w/ @andymcc, he was seeing the reverse (short names rather than FQDNs).  Upon further investigation, the hostname didn't appear to be set correctly on my AIO (the FQDN was listed in /etc/hostname and a hostname -s was returning the FQDN), where-as on @andymcc's installation everything looked correct.  I will assume that the QE team's hosts were originally configured incorrectly which is why they requested that we adjust these plays.
I am currently looking at ways to handle both situations so that we don't have to go back to deployers and tell them to fix their hostnames (as ideal as that would be).

Need to determine if this will be an icehouse backport.
@claco @mancdaz @cloudnull

Backport to juno handled in https://review.openstack.org/#/c/169668/.

This ended up being released in 10.1.3 here:
https://bugs.launchpad.net/openstack-ansible/+bug/1439083


NIFI-2208 - Support Custom Properties in Expression Language (v1.0.0)

This is the commit of the custom property expression language support with Variable Registry (for master)

Hey @YolandaMDavis, looks like this has merge conflicts. Could you update the PR to the latest master?

I will review this.

thanks @alopresto I just rebased with the latest to remove conflicts

@alopresto here is a write up of use case and demo of example flow (also refers to a gist with files from demo)
Supporting Custom Properties for Expression Languages.docx

@YolandaMDavis, what happens if an incoming FlowFile, FileVariableRegistry and custom EL attributes all have the same key? Is there any ordering enforced? From looking at the code, I believe it to be random.

Is there any documentation added to the Admin/Users guide on how properties in a file are exposed via EL?

@JPercivall thanks for the review!  Just wanted to add a couple of notes on questions you had above:

@YolandaMDavis, what happens if an incoming FlowFile, FileVariableRegistry and custom EL attributes all have the same key? Is there any ordering enforced? From looking at the code, I believe it to be random.

The order is dictated through the MultiMap's (now ImmutableMultiMap) use of an ArrayList internally which maintains the order in which a map was added to that list.  In the case you mentioned where the same key name appears across a flow file, file/property registry etc the first map encountered in the ImmutableMultiMap's list with that value will be used.
When the variable registry is initialized via VariableRegistryUtils (as it is in most of the application) environment variables and java application properties are added first, with those values taking precedence.
The details of this will be added to the admin guide (which I believe is part of another issue on Jira list, but I can confirm).

Yup let's make another ticket for the documentation and just address what was already merged into 0.x
+1
Visually verified code and did a contrib check build. In standalone and a secure cluster I used the below template (rename to xml) to test various levels of the EL access and all performed as expected. Thanks @YolandaMDavis, I will merge it in.
Variable_Registry_test.txt

Whops accidentally mistyped the close statement but I merged it in[1]. @YolandaMDavis could you close this?
[1] https://github.com/apache/nifi/commit/8412d2662ae1cab6c52be62272a2971d199719df

Thanks so much @JPercivall ! Will take care of closing.


Not able to download files and getting the following error in socket creation

Description of the bug
I am getting the following error in multiple commands.
To Reproduce
Steps to reproduce the behavior:

Ran the command pytr -v debug dl_docs 'C:\Users\XXX\Downloads'
Followed the instructions

Expected behavior
Files to be downloaded.
Error log
2024-11-10 16:56:38+0100 main      DEBUG    logging is set to debug
2024-11-10 16:56:38+0100 account   INFO     Credentials file not found
Please enter your TradeRepublic phone number in the format +4912345678:
xxx
Please enter your TradeRepublic pin:
Pin (Input is hidden): xxx
Save credentials? Type "y" to save credentials:
2024-11-10 16:56:41+0100 account   INFO     Credentials not saved
Enter the code you received to your mobile app as a notification.
Enter nothing if you want to receive the (same) code as SMS. (Countdown: 31)
Code: xxx
2024-11-10 16:56:46+0100 account   INFO     Logged in
2024-11-10 16:56:46+0100 dl        INFO     Created history file
2024-11-10 16:56:46+0100 timeline  INFO     Subscribing to #1 timeline transactions
2024-11-10 16:56:46+0100 api       INFO     Connecting to websocket ...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "c:\users\xxx\.local\bin\pytr.exe\__main__.py", line 7, in <module>
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\pytr\main.py", line 266, in main
    asyncio.get_event_loop().run_until_complete(dl.dl_loop())
  File "C:\Users\xxx\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\pytr\dl.py", line 68, in dl_loop
    await self.tl.get_next_timeline_transactions()
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\pytr\timeline.py", line 30, in get_next_timeline_transactions
    await self.tr.timeline_transactions()
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\pytr\api.py", line 506, in timeline_transactions
    return await self.subscribe({"type": "timelineTransactions", "after": after})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\pytr\api.py", line 322, in subscribe
    ws = await self._get_ws()
         ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\pytr\api.py", line 301, in _get_ws
    self._ws = await websockets.connect(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\websockets\asyncio\client.py", line 441, in __await_impl__
    self.connection = await self.create_connection()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\pipx\venvs\pytr\Lib\site-packages\websockets\asyncio\client.py", line 367, in create_connection
    _, connection = await loop.create_connection(factory, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: BaseEventLoop.create_connection() got an unexpected keyword argument 'extra_headers'

Environment

OS: windows
pytr version: 0.3.1
Python: 3.11

I had the same error running this command on my Mac with Python 3.12.4.
Could solve this issue by running the program in a virtual python 3.9 environment (see #141).

Did not work for me with Python 3.9.13 either.

I asked ChatGPT for help, and it suggested downgrading the WebSockets version:
pip install websockets==10.2
Took me also some time to get Python 3.9 running in a virtual environment (venv).

Thanks a lot. worked for me too with the downgraded websockets version.

Just for others, not sure if that is the best solution but it works for me, didn't work much with python yet:

installing pytr with python 3.9 pipx reinstall pytr --python /opt/homebrew/bin/python3.9
downgrade websockets pipx inject pytr websockets==10.2 --force

Just for others, what worked for me under Windows 11

installed python 3.12
mkdir pytr
cd pytr
python -m venv pytr
pytr\Scripts\activate
pip install pytr
pip install websockets==10.2
pytr dl_docs -n +49xxxxx -p xxxx C:\Users\xxxx\pytr

Using it on Ubuntu WSL (Ubuntu 20.04.6 LTS (GNU/Linux <IP_ADDRESS>-microsoft-standard-WSL2 x86_64)). Python 3.13. Same error as this.
Downgrading websockets as suggested here works
pipx inject pytr websockets==10.2 --force

Without reinstallation, just force downgrade of websockets package on the venv

Same Problem with 3.13.1.
websockets downgrade worked for me on win11

I was using this Dockerfile to build my own local container when I was hit with this error. I read all the tips here and some other related issues on thus project and am still failing. Furthermore, I'd appreciate some pointers.
I went with the advice from this thread to go with python 3.9 and websockets==10.2 and no such luck. What happens is, I get asked the PIN, the connection to TR succeeds, I can see the subscriptions happening and downloading, and it gets to about 5200 /out of 5500 and then hundreds of error messages like this one:
16:28:21 Received error message: '6299 E {"errors":[{"errorCode":"UNKNOWN_ERROR","errorField":null,"errorMessage":"Unknown error","meta":{"source":"PLATFORM"}}]}'
16:28:21 Error response for subscription "{'type': 'timelineDetailV2', 'id': '5de9b8d1-ae18-4dfb-b01d-714d13ea1f6a'}". Re-subscribing...

FROM        python:3.9-alpine

ARG         USER=xyz
ARG         UID=1000
ARG         GID=1000

RUN         python3 -m venv /app &&\
            /app/bin/pip install pytr websockets==10.2 &&\
            addgroup -g ${GID} -S ${USER} &&\
            adduser -u ${UID} -S ${USER} -G ${USER} &&\
            chown -R ${UID}:${GID} /app

WORKDIR     /app/bin

USER        xyz

CMD         ["--help"]

ENTRYPOINT  ["/app/bin/python3", "pytr"]

Just in case you're curious, I override the CMD in the compose file with:
    command:
      - "dl_docs"
      - "/documents"

and /documents is a mounted volume.

Good morning, is there any new information about this bug? I have tried to follow the instructions and tried everything but no improvement.
Phyton version is 3.12.4
pytr 0.3.1
Have tried with websocket 10.2 but all came up with the same error.
Thanks in advance for any new ideas
09:08:46 All timeline details requested
09:08:46   1/849: Netflix -- Verkaufsorder - 2024-12-09T07:31:42
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\Martin\pytr\pytr\Scripts\pytr.exe_main.py", line 7, in
File "C:\Users\Martin\pytr\pytr\Lib\site-packages\pytr\main.py", line 266, in main
asyncio.get_event_loop().run_until_complete(dl.dl_loop())
File "C:\Users\Martin\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "C:\Users\Martin\pytr\pytr\Lib\site-packages\pytr\dl.py", line 85, in dl_loop
await self.tl.process_timelineDetail(response, self)
File "C:\Users\Martin\pytr\pytr\Lib\site-packages\pytr\timeline.py", line 130, in process_timelineDetail
event = self.timeline_events[response["id"]]
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'a796499a-8148-4753-b5e2-46f636ec804e'


Google Apps Script - Problems parsing JSON

Hi!
I am trying to update a GitHub json file from Google Apps Script (using UrlFetchApp.fetch() and I am getting the following error
{"message":"Problems parsing JSON","documentation_url":"https://developer.github.com/v3/repos/contents/#update-a-file"}

The parameters are the following:
var params = {
    "message": "This is  a test",
    "committer": {
      "name": "Daniel Deicide",
      "email"<EMAIL_ADDRESS>    },
    "content": "aGVsbG8sIHdvcmxkCg==", // this has to be Base64 encoded
    "sha": getSHA() //this is a function that returns sha of the file I want to update
  };

  var options = {
    'method' : 'put',
    'payload' : params
  };

Could you please have a look and help me out with this?
Thank you!

It looks like the request to the GitHub API is not correct.
Did you check the link the error gave?
https://developer.github.com/v3/repos/contents/#update-a-file

Yes, I tried with Postman (same setup) and it works

I fixed it adding JSON.stringify() to params.
I will close the issue. Thank you for your time.


Seplos v3 sensor information

I've noticed on v3 some of the sensors have the wrong register type. Some are set as holding - where it should be Read.
I also noticed the tempreture sensors aren't reading the correct temperatures. They seems to be about 5 degrees higher than the temperatures shown on the actual BMS display?

Could you enable the debug output of the uart component and provide some logs? Because of the debug mode they will include the RAW traffic and I will able to replay the communication locally.

OK great. Attached is the debug file. I think temperature readings are the PCB temps. At the time of this debub the Cell temps were all around 23degrees C
seplos-esp_logs.txt

@syssi I have noticed all the temperature readings are wrong and they only fluctuate within 1°c no matter the actual temps.

Ok I have it figured out now. Seplos use a calculation for temps. It's not just the sensor readings. It's the sensor reading minus 2731. See sample sensor feed from the seplos modbus document. So to get correct temps use filters:
lambda: return x / 10 - 273.15;

Do you like to provide a pull request to contribute your finding?

You could apply a negative offset filter before the multiply filter.

I'm very very new to home assistant and yaml. I just asked chat gpt fix my code to deduct 2731 off the sensor readings  and make it work that it has the correct decimal placing🤣 I don't know how pull requests work in GitHub either!! A negative offset filter before the multiply filter should work as well. If you give me the code I'll try it out for you

the cell temperatures are in Kelvin, the register answer in 0.1K.
Kelvin to Celsius: C = K - 273 (C = K - 273.15 if you want to be more precise)
Kelvin to Fahrenheit: F = 9/5(K - 273) + 32 or F = 1.8(K - 273) + 32.

I will update the configuration example tomorrow. I'm happy about some testing later on!

I've noticed on v3 some of the sensors have the wrong register type. Some are set as holding - where it should be Read.

Fixed.

I also noticed the tempreture sensors aren't reading the correct temperatures.

Fixed.
Please give it a try using the updated example configuration. Feel free to create more issues if you miss something!


Implement peek(int nth)

The method should read the nth element from the queue without changing the queue.
A potential use case of this method could be to read a portion from the queue in a loop and process it in a streaming API manner. If and only if all of the portion elements are processed successfully then the whole portion is removed from the queue.

Do you have an actual use case for this? We don't want to add an API for a
purely hypothetical use case.
On Tue, Oct 17, 2017, 9:39 AM aacic<EMAIL_ADDRESS>wrote:

The method should read the nth element from the queue without changing the
queue.
A potential use case of this method could be to read a portion from the
queue in a loop and process it in a streaming API manner. If and only if
all of the portion elements are processed successfully then the whole
portion is removed from the queue.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/square/tape/issues/174, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEEEc-qphWkjJ0UfTsxEgUSQbFj8oPnks5stK4sgaJpZM4P8K3D
.

The hypothetical use case stated sounds like the existing APIs:
List<Data> section = queue.peek(num);
if (processSuccessfully(section)) {
  queue.pop(num);
}

Is this instead a request for Data nthElement = queue.peekBackInQueue(n)? I'm not sure what the use case would be, but could you do it in your application code by peeking n elements in the list, and getting the last element?

If the request is for getting the first n elements (and not specifically the nth element only); then the iterators can be used pretty efficiently as they're lazy.
    int i = 0;
    for (byte[] bytes : queueFile) {
      if (i++ > n) {
        break;
      }
      process(bytes);
    }

peek(n) doesn't allow for minimal memory usage and the iterator doesn't allow for concurrent modification.

A potential use case would look like this:
int n =  10 //  number of elements for procesing
for (int i = 0; i < n; i++) {
Element  element  = queue.peek(i); // reads the ith element
StreamingAPIProcessor.process(element);
}
if (StreamingAPIProcessor.isSuccessful()) {
queue.remove(n);
}
Given that there is a queue of 10 000 elements. And we want to process 1000 element as a portion (e.g. upload it to the server.). If we process 1000 element in the streaming manner we'll consume less memory. We can't use the iterator because it doesn't allow concurrent modifications.

I don't understand the example you shared fully (StreamingAPIProcessor processes one element at a time but reports status about n elements?). But anyway, that example could be written simply as:
    int n = 10;
    int i = 0;
    for (byte[] bytes : queueFile) {
      if (i++ > n) {
        break;
      }
       StreamingAPIProcessor.process(bytes);
    }

 if (StreamingAPIProcessor.isSuccessful()) {
   queue.remove(n);
}

I'm not sure where the question of concurrent modification comes in here - your example is synchronous, you're processing n elements one at a time, and removing n elements when all are done.

StreamingAPIProcessor is e.g. uploading n elements to the server using e.g. JsonGenerator (Jackson Streaming API). So, if the network request is successful we remove n elements. The idea is not to load n elements into the memory but to read only one element at a time.
In general, the system generates new elements and puts them into the queue (on some other threads). So that's why we can't use the iterator.

So one has to provide some pseudo code with a concurrent access adding stuff to the queue just so you accept it as a valid usecase? Oh well
QueueFile queue = new QueueFile()
new Thread() {
  run() {
    synchronized(queue) { queue.add(rand()) }
    sleep(rand())
  }
}

while (!queue.isEmpty()) {
  batchSize = Math.min(queue.size(), MAX_BATCH_SIZE)
  for (int i = 0; i < batchSize; i++) {
    synchronized(queue) { item = queue.get(i) }
    stream(item)
  }
  if (isBatchSuccessfull()) {
    synchronized(queue) { queue.remove(batchSize) }
  }
}

Took me a while to realize that the iterator() is not available in the version of the documentation.
<dependency>
    <groupId>com.squareup</groupId>
    <artifactId>tape</artifactId>
    <version>1.2.3</version>
</dependency>


fix(channels): filter channel list on node alias

Update channel filter handlers to correctly consider the node alias.
Fix #396

@Empact could you give a re-review please? I've cleaned this up and simplified things a little.

utACK 8d448d7

Thanks @Empact - I've updated to remove the lodash dependency.

re-utACK e8e6e9f


feat(util): Add isHash utility

isHash is a utility function that checks if something is an object,
but is neither an array nor a function.
Closes #110

Coverage remained the same at 100.0% when pulling cee420a390e00ce0608d42e3e21b7b63c76bd6a8 on jpbufe3:feat/110-ishash into 7a4f3d0fbd1ffc80bc831582b5a579dcdb1e7ca7 on ansman:master.

Thanks

This is now released in 0.10.0


init.lua: remove 'legacy' tag from fidget.nvim

Since https://github.com/j-hui/fidget.nvim/pull/143, the legacy tag is no longer necessary (or recommended) for my plugin.
(Then again, I'm not sure if the point of including fidget.nvim here was to be instructive about how Lazy tags work, so maybe this PR defeats the point?)

I thought fidget was included because we wanted its functionality? Happy to merge this in any case, but if you think fidget should be removed from kickstart please open a separate issue and we can discuss there.

What I meant was, I wasn't sure if fidget.nvim was included for the sole purpose of demonstrating how Lazy's tag mechanism worked---in which case, this PR would be counter-productive.
But that doesn't seem to be case. I'm very happy to have Fidget included in kickstart (: Thanks for merging!


Browser native autocomplete dialog showing up sometimes

Since the target input doesn't have autocomplete="off" set, because it overrides this attribute somehow, two autocomplete menus are showing up, which is not really wanted. Maybe an option to set autocomplete="off" could fix this issue.

autocomplete off doesn't work in most situation
this is why i'm using a random string for it

do you have an example and a given browser that show this issue?

mhh that's odd, i was using "off" before but sometimes it was ignored by the browser. but it seems my method doesn't work any better so maybe better to stick with off. you can check on master

@AgentSmith0 fyi i added a new option (the original reason why i didn't use autocomplete="off") for fields like city/addresses where browser native autocompletion may show up
https://github.com/lekoala/bootstrap5-autocomplete/releases/tag/1.1.20


Check NODE to differentiate the different partitions

https://github.com/broker-game/git-broker-client/pull/108


data-models

Please use the form below, leaving the prefilled data to help us. Thank you.
Page link: data-models
Document link: _index.md
My Issue/Suggestion
i'm working in samsung sds by using Mendix system. and i think mendix documents need "example code" for developer
not only me, every my collegues said  Mendix should add example code in mendix documents.
i hope this issue is improved ! thank you

Hi @cyeongb thanks for the feedback. We do have code samples in docs where those are required and useful, and we're always looking for feedback on docs where code samples could be added or improved. Do you have any such specific examples? Otherwise, Mendix as a low-code platform limits the need for code samples in our docs, as we do forefront visual modeling. Please let me know if you have some specific doc examples here where code samples may be useful.
Thanks.


Importing only the extract utilities

Hello @adbar, thanks for your tremendous work on the library. Do you know if there is a way to install and then import the library so that you will only load the utilities related to raw content extraction from a html string? If not, is there anyway we can discuss this particular topic and see if I could help you implement this in any way? My use case is basically the following: I have a CLI tool that currently relies on dragnet and I would like to jump ship and adopt trafilatura. My issue is that I don't want to install the net-related dependencies you list in your setup.py (notably requests and tldextract) because they will clash with some of my dependencies and I have my own means of downloading things, dealing with urls etc.
Have a good day,

Hi, thanks for your interest. There is no easy way to specify a subset of dependencies in setup.py and I know quite a few users who rely on the integrated download function so I'm not keen on making it optional. If I understand properly you'd only use the bare_extraction function? Here are the options I see:

I'm also thinking about replacing tldextract, maybe the solution you use is open-source and more efficient? In that case I'd like to know more about it ;)
We could integrate the extraction into some kind of http.server which would run within its own virtual environment (but it wouldn't be compatible with a CLI tool I guess?)
Since you would execute the downloads separately it would be possible for you to install all packages but requests and tldextract (with pip install --no-deps or pip uninstall -y requests, tldextract afterwards), we could wrap the few functions that rely on it with try: / except NameError: and you would keep your CLI tool from complaining about missing packages. A bit ugly but probably easy.
As a last resort we could provide a default installation pip install trafilatura[net] but that would mean doing quite a lot of plumbing on the internal functions.

What do you think?

Hello @adbar, thanks for the answer.

If I understand properly you'd only use the bare_extraction function?

Exactly, I only need those core  utilities.

I'm also thinking about replacing tldextract, maybe the solution you use is open-source and more efficient? In that case I'd like to know more about it ;)

I am currently relying on the tld python library, itself relying on this public facing list of TLDs (but you can also override it if required). Then I have my own routines similar to courlan there.
The only way I can think of to keep both current version non breaking for its current users and not requiring complex installation schemes is unfortunately to create a separate pypi package like trafilatura-extract for instance, that this package would depend on but I can see how this would be bothersome for you to do that.
For reference, the CLI tool I am speaking about is this tool called minet and developed by my research lab: https://github.com/medialab/minet

Hi, thanks for the links, it seems like we're working on similar topics, nice!
Replacing tldextract shouldn't be an issue here, tld seems to be a good candidate. I had it on my list anyway.
I think I could replace requests as well or at least make it optional by providing another download function. What is the dependency clash you face? And could you please point me to the place in your code where you perform the downloads?

Replacing tldextract shouldn't be an issue here, tld seems to be a good candidate. I had it on my list anyway.

What's the issues you are having with tldextract, just for the reference? On a side note, do we agree that the core extraction schemes don't require to have any knowledge of tld data?

What is the dependency clash you face?

Clash is maybe a strong word. I don't think installing your lib would crash the installation of some other dependencies I might have. My issue here is more down the line when I finally compile the tool as a standalone executable using pyinstaller or pyoxidizer because I cannot easily tell them not to package some dependencies that will not be actually used and this translates to longer starting times and larger binary sizes.

And could you please point me to the place in your code where you perform the downloads?

I currently use urllib3 (which is used by requests under the hood) but cannot rely on requests for performance and multithreading reasons. I will probably switch to pycurl in the future, though. I cannot point you towards the specific code because the codebase is quite large and the related code is not just a single function but most of it can be found here and here.
What I could do, as a first step, would be to switch from dragnet to trafilatura as is and see how much cruft it does entail regarding packaging/compilation and come back if there are "real" issues I cannot circumvent.

There is not real issue with tldextract but your original question is totally relevant, it's generally a good idea to reduce the total number of package dependencies. Since tld is self-sufficient it would be a good replacement.
I agree, the performance of requests is suboptimal. I'd recommend pycurl but didn't ship it with trafilatura since its dependence on libcurl doesn't make it so portable or easy to install.
OK, please try to use the package as is, I'm ready to help if there are packaging or compilation issues arise.

The download utility wasn't so complex anyway, I adapted it to use urllib3 directly in 273a319c1199d32f964d5402195dcb64d34a415f, so requests will be dropped from setup once the next version is released. I had to keep chardet though (with support for cchardet), just so you know.
Since tld only supports Python 3.6 upwards I'm going to wait a little to replace tldextract with it, just tell me if you run into problems.

Hello @adbar. I also use cchardet on my end anyway.

Since tld only supports Python 3.6 upwards I'm going to wait a little to replace tldextract with it, just tell me if you run into problems.

No problem so far apart from needing to load a suffix list twice once for tld and tldextract in my pyinstaller compilation but nothing critical.
That's weird it does not handle < 3.6 since it supports 2.7 though :)

The new package versions are out, requests has been dropped out.

The new package versions are out, requests has been dropped out.


Add code to review

I have copied and uploaded all the todo list code for review

Well Done @shakerAbuDrais! You have done a great job on this project

Your code fulfills all three DRY, KISS, and YAGNI rules, which means, it isn't repetitive, your code is useful and you're not making use of code that you do not really need for the project, well done, keep going!


It doesn't work in Firefox

You can just check your demo in FF36.

It works if we comment the line
"touchDevice = 'ontouchstart' in document.documentElement"

Thanks for reporting this @smartcorestudio. Sorry for the delay. I just tested in FF39 for Mac OS X and it worked. Do you think it’s possible https://github.com/HubSpot/sortable/issues/23 was the issue? (Were you on a 2-in-1 or tablet running Firefox?) Thanks in advance.

Assuming this was #23, that was fixed with cab1613b721e7a7512417ded8fb62d163c647777. Please let us know though if your issue isn’t fixed with by cab1613b721e7a7512417ded8fb62d163c647777 and we’ll reopen. Thanks!


Navigate via context menu

Since my previous PR caused selection issues, this adds the same navigation function to artist/album views via the right-click context menu instead.

This is a great idea. I'd suggest the following order (and letter case) though: Play/Pause -> Go to Artist -> Go to Album ->  Add To -> Edit.

I've corrected the order.

Yes thanks, but the letter case is still wrong.

D'oh! Sorry. Fixed now.

Thanks!


Angular ui-select not displaying

I want to use ui-select, but it is not displaying anything and there are no errors in the developer console. I've been googling this for a few hours now and I've tried all the suggestions I've found, but nothing has helped.
I use Chrome and I've included Bootstrap version 3.3.4 and Angular version 1.4.8 in my SPA project. I've also included the latest versions of select.min.css and select.min.js. I added ui-select (version 0.19.5) to my project through NuGet in Visual Studio 2015.
I've reduced my code to a very simple example while trying to isolate the problem. Here is what I have right now (this is in my view):
'

    <ui-select ng-maxlength="10">
<ui-select-choices repeat="f in ['light','hpexp']">
{{ f }}
</ui-select-choices>
</ui-select>

'
I've tried adding a ng-hide, like so, to no avail:
'

    <ui-select ng-maxlength="10">
<ui-select-choices repeat="f in ['light','hpexp']" ng-hide="!$select.open">
{{ f }}
</ui-select-choices>
</ui-select>

'
Here is what it looks like in the page. The div where the drop-down should be displayed is there, but it's height is 1. Also, as you can see, there are no errors in the console:

What could be the problem here?

I don't see a ng-model on your ui-select.
Maybe instead of {{f}} try

Hi, @lvasilev1  Can you make a example plunker for this? People can then have a better look at your issue. For now I don't see any problems , did you forgot to include the lib?

@ckosloski @Jefiozie Yea, I didn't have ng-model on it, cause I thought that was not necessary if you just wanted to display some values and not tie them to an object in the controller.
So I added that and also in my mainApp I didn't have dependencies on "ui.select" and "ngSanitize". Once I did all that and also included the angular-sanitize.min.js in my index Html file, things started working.
Thanks!


LFS and BIG FILES

Does this support lfs?

https://github.com/caprover/deploy-from-github?tab=readme-ov-file#example-1---deploy-using-image
This method simply build the image on Github so you don't even need Git LFS to push to anywhere.


Grey-out buttons after submission

#456 # @olliefreeman @pjmonks

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Shikha Vani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Ok. Sure. I will go through it and will update you
On Sat, Feb 5, 2022 at 8:39 PM Peter Monks @.***> wrote:

@.**** requested changes on this pull request.
Generally the solution looks good, thank you. Please could you review my
comments though to fix up a couple of things.
Also, there are linting issues found in your changes which our build
picked up on. If you run this command you will get a HTML report, you'll
only need to care about the errors found (there will be lots of warnings,
please ignore those):
npm run eslint -- --format html -o eslint_report.html
@jamesrwelch https://github.com/jamesrwelch You've mentioned that there
may be more places where a submission button should be disabled whilst
saving, perhaps the issue should be updated to list these places to clearly
identify them? Then this PR could cover all those cases too.
In src/app/wizards/dataModel/data-model-main/data-model-main.component.ts
https://github.com/MauroDataMapper/mdm-ui/pull/457#discussion_r800065904
:

@@ -51,6 +51,7 @@ export class DataModelMainComponent implements OnInit {
private folders: FolderService,
private title: Title) { }

disabledSave = false;

Could this be moved to above the constructor please? So directly below the
field parentFolder? This will keep all fields organised the same way.
Also, I would suggest renaming this to isSaving to give more semantic
meaning.
In src/app/wizards/dataModel/data-model-main/data-model-main.component.ts
https://github.com/MauroDataMapper/mdm-ui/pull/457#discussion_r800066261
:

     this.stateHandler.Go('datamodel', { id: response.body.id }, { reload: true, location: true });

 }, error => {

   this.disabledSave =  false;

There is a shorter way of setting this flag back to false without having
to duplicate it. There is an rxjs operator called finalize() which, if
included in the pipe(), will always be called regardless of whether it
was successful or not. Including something like the following would then
you only need to set this flag once:
.addToFolder().pipe(
catchError(error => {
// Error case
}),
finalize(() => this.disabledSave = false)  // Always called at the end).subscribe((response) => {
// Success case});
—
Reply to this email directly, view it on GitHub
https://github.com/MauroDataMapper/mdm-ui/pull/457#pullrequestreview-873940242,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPSN4RLOCKSOREIFPQEWKDUZU4RXANCNFSM5NMEJK4Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

Thanks for your PR @shikhavani. Sorry for the delay, we'd missed this update. Would you be able to sign the Contributor Licence Agreement for this to be merged?
Link to the CLA: https://cla-assistant.io/MauroDataMapper/mdm-ui?pullRequest=457
cc @jamesrwelch


Scaling for factor graphs that are 100TB

We've encountered grounding results that Zifei had to run with a 900GB-RAM machine because the 300GB-RAM machine wasn't big enough for the factor graph.
But the factor graph consists of millions of small connected components (say one per document), so we should need no more than 64KB of memory if we process the components one by one (not including space for the model). In fact, a lot of existing programs have graphs consisting of singleton variables...
We don't even need to detect those components -- this pattern is so common that we could introduce some syntax in DDLog to provide hints; e.g., telling DD that doc_id can be used as a partition key.
Resolves #256

This was a very old feature... where did this go @zhangce @netj @SenWu
You don't even need syntax for this, it's easy to detect... @feiranwang

I will sync with Zifei and push this feature in another branch.

Zifei is out of the country.  :)
Is this code stable? I thought you built this months ago! Has it been code reviewed?
Also, @netj and @feiranwang, it should be super easy to declare categorical that are linked (say for our old entity linking design, they need to be able to express
LinksToOne(candidate, entity) as a categorical random variable (each candidate maps to one entity).
This should be super easy (@key is fine). @zhangce, I know you're traveling but your thoughts are welcome when you get back on line :)
@ajratner I don't want to redo the full constraints--no one seem to use them, and they have lots of code complexity... maybe in the summer :)

I am not sure it is stable. I only test it on small examples (around 100k docs). But this is really a good opportunity to test my code and put it in. I am talking to Feng about the case. :)

Ha, OK :)

Sen, how are we doing here?

Yes, I have already synced with Feng and I will add partition support based on DD v0.8. And I briefly talked with Jaeho about it yesterday and we will come up a concrete plan for it on Wednesday.

@SenWu Any update?
We have an app with 100K docs, 13M vars, 42M weights, 248M edges, and 242M factors. The machine has 2 NUMA nodes. DD was stuck at CREATE FG ON NODE ...1 for over an hour with the sampler using 70GB of RAM and the machine apparently thrashing...
2016-02-23 17:58:50.972139 ################################################
2016-02-23 17:58:50.972150 # nvar               : 13447656
2016-02-23 17:58:50.972162 # nfac               : 242889469
2016-02-23 17:58:50.972175 # nweight            : 42308790
2016-02-23 17:58:50.972187 # nedge              : 248001424
2016-02-23 17:58:50.972200 ################################################
2016-02-23 18:02:27.024966 LOADED VARIABLES: #13447656
2016-02-23 18:02:27.025074          N_QUERY: #11900206
2016-02-23 18:02:27.025096          N_EVID : #1547450
2016-02-23 18:02:47.727552 LOADED WEIGHTS: #42308790
2016-02-23 18:05:38.553535 LOADED FACTORS: #242889469
2016-02-23 18:09:43.023361 CREATE FG ON NODE ...1

@BrettMeehan is making progress now. Brett, could you give us some updates here? Thanks.
BTW: for large factor graph, I suggest using only 1 numa node(the sampler argument is -c 1).

Yes, the sampler uses 40GB of memory with -c 1 and works for now. But we have at least 50X as many docs. So partitioning is critical.

How did we lose the ability to bring connected components into memory?
This needs to be fixed soon... Let's talk about this in dev meeting.
On Tue, Feb 23, 2016 at 11:43 PM alldefector<EMAIL_ADDRESS>wrote:

Yes, the sampler uses 40GB of memory with -c 1 and works for now. But we
have at least 50X as many docs. So partitioning is critical.
—
Reply to this email directly or view it on GitHub
https://github.com/HazyResearch/deepdive/issues/478#issuecomment-188128352
.

@SenWu I'm curently working on adding checkpointing to the learning and inference steps of the Gibbs sampler so the application can be paused and resumed. Also browsing Feiran's partition code from 1 year ago to see what parts we can merge to the main branch.