Support Markdown Extension Configs

Mkdocs uses Python-Markdown and offers support for users to specify extensions supported by Python-Markdown. However, currently mkdocs provides no way (sort-of; see below) to pass config settings into the extensions specified. For example, I would often like to use the TOC Extension's permalink feature to add the paragraph symbol (¶ or ¶) to the end of each header (h1-6) as a self-link. A very handy feature in documentation (in fact, it's used by Python-Markdown's docs, which I've linked to). Another use may be to customize how syntax highlighting works. Or non-english users may want to override the default substitutions used by SmartyPants. etc, etc...
Sure, mkdocs could hard-code some specific settings for things that make sense. Granted, not all extensions configs make sense in the context of mkdocs. But do you really want to be fielding bug reports for years to come asking for some additional config setting? And don't forget about third party extensions. Some of them (like a few of the math extensions) make a lot of sense for some users of mkdocs. But you couldn't possibly expect to provide good support for them without offering a holistic approach.
Interestingly, as mkdocs' config file is YAML and Python-Markdown's extension_configs keyword accepts a dictionary of dictionaries, it is pretty easy to replicate in the config file. Of note is the recent addition to Python-Markdown's CLI of support for a YAML config file for this very thing. One difference I would suggest is that, unlike the CLI feature, there should be no need to also provide a list of extensions separate from the extension configs.  I would expect the implementation to be something like this:
if isinstance(config['markdown_extensions'], dict):
    extensions = config['markdown_extensions'].keys()
    extensions_configs = config['markdown_extensions']
else:
    # backward compat with old list type
    extensions = config['markdown_extensions']
    extensions_configs = {}

# then later in the code...
md = markdown.Markdown(
    extensions=extensions,
    extensions_configs = extensions_configs
)
html_content = md.convert(markdown_source)

Note that support for the existing list of extensions is still supported. The new behavior only happens when the config setting is of the dict type. And the new format in the config file would look something like this:
markdown_extensions:
    markdown.extensions.toc:
        permalink: True
    markdown.extensions.codehilite:
        linenums: True
    markdown.extensions.smarty:
        smart_angled_quotes: True,
        substitutions:
            left-single-quote: '‚',
            right-single-quote: '‘',
            left-double-quote: '„',
            right-double-quote: '“'

A few notes about this proposal:

Actually mkdocs does provide a method of sorts for passing in config settings which involves including the settings right in the name of the extension (see the old docs). However, that behavior is deprecated in Python-Markdown 2.6 (released yesterday) and was "pending deprecation" in 2.5. Even in older versions, it is very limited as no spaces are allowed and only string types could be used (booleans were tricky -- usually a 0 or 1 and then special code in the extension to call int() on the string).

In previous versions of Python-Markdown (prior to version 2.5), the extensions_configs keyword expected a list of tuples for each extension rather than a dictionary (again see the old docs). That being the case, the earliest version of Python-Markdown this would work with is 2.5.

As of today, mkdocs only supports Python-Markdown version 2.4 as that was the last version to support Python 2.6 (see #165). However, dropping support for Python 2.6 appears to be earmarked for mkdoc 1.0.0 (0.12.0 is currently in development with 1.0.0 being the next milestone).

Given the above, I expect that this feature would be added to 1.0 at the earliest (although I understand if the desire is to not add any new features until after 1.0). Given that fact, I don't see much point in actually creating a pull request just yet (with tests and docs -- the code changes are pretty simple -- see above) as I'm not really interested in continually rebasing it until you guys are ready to accept it. That said, if the idea is accepted, I'll happily do the work when the time comes. If, on the other hand, you guys are not interested in adding support for this, I'll probably look for another project which fits my needs better (not having this is a non-starter for me).

Sorry for the slow reply to this well detailed post. This is quite an intersting one, I have wanted to pass config to the extensions too (also for permalink as it happens) but I couldn't figure out a nice way to do it. It looks like your method would work, but it is rather huge :) This is perhaps one of the reasons Tom was hessitant to let users do this at all.
That being said, I think it is probably an okay approach, and I'd be +1. You are right, it does sound like we need to do this for the 1.0 milestone (with the python 2.6 drop). Given it is a fairly small change to an existing feature I'd be okay with it being in the 1.0 milestone and not after. Also, it'd be awesome in Python-Markdown used MkDocs :smile:

What would the YAML look like if you had some extensions with config, and some without?

@d0ugal that a good question. As it turns out, YAML is pretty flexable.  So you can use JSON (with curly brackets) as well. And as Python Dicts use the same format as JSON check this out:
>>> d = yaml.load(config)
>>> s = str(d)
>>> s
"{'markdown_extensions': 'codehilite': {'linenums': True}, 'toc': {'permalink': True}, 'meta': None, 'extra': {}}}"
>>> yaml.load(s)
{'markdown_extensions': 'codehilite': {'linenums': True}, 'toc': {'permalink': True}, 'meta': None, 'extra': {}}}

Above, I created a string of the Python dict (which has no line breaks but full use of brackets) and passed that back to YAML and it gave back a dict.
Technically, this is an easy problem to solve. However, the real issue us making sure the format is easy for users who are unfamiliar with YAML, JSON, and/or Python Dicts. That is probably more of a documentation issue. How do we want to document mixing extensions with and without config options?

huh, interesting, I didn't realise that "meta: " was a valid line, for some reason I expected the value to be required.
As for how we want to document it, I'm not sure - but I'm not a fan of the approach in https://github.com/tomchristie/mkdocs/pull/366 :)

I agree. As I noted in a comment on that PR, that is a deprecated feature which will stop working in the next release. I was referring to this in my note 1 in my proposal above.

So I was looking as the mkdocs source to determine where to implement this and I'm not sure where you would prefer this be done. Basically, I'm looking for where I should insert this code (or some variation thereof):
if isinstance(config['markdown_extensions'], dict):
    extensions = config['markdown_extensions'].keys()
    extensions_configs = config['markdown_extensions']
else:
    # backward compat with old list type
    extensions = config['markdown_extensions']
    extensions_configs = {}

My initial thought was to simply add this to mkdocs.build.convert_markdown which would work fine and be the least intrusive. However, I noted that there is the mkdocs.config.validate_config function which seems like it might be the proper place to do this. However, to do that, then the config (at least internally) would need to also have a markdown_extension_configs key which would need to be passed to mkdocs.build.convert_markdown from mkdocs.build._build_page. Of course, mkdocs.build.convert_markdown would also need to have an additional keyword argument to accept the optional extension_configs. All doable, but more intrusive and slightly changes the API of mkdocs.build.convert_markdown. It just feels more "correct" that way to me.
I note that the internal extensions are all defined within mkdocs.build.convert_markdown. It would seem to me that that should all be happening in mkdocs.config.validate_config as well... which is why I'm having this quandary. In any event, I suppose the extension config stuff should be in the same place as the internal extensions.

Yup, I agree. This should all happen in validate_config (including the current defaults which are not.) So I would be happy to see the new and old relevant code move there. The API changes shouldn't be a problem other than for the existing tests and we don't consider our the code to be a public API.
My only concern is that validate_config is already becoming a nightmare to maintain. I've been looking for a way to better manage that part of the code, but I've not had any particularly good ideas yet. Just splitting it into many smaller methods that validate different areas might be enough.

Moving the extension stuff to validate_config is easy enough. But most of the tests in tests/build_tests.py fail without creating and passing a config instance in. While it is certainly doable, it makes the build tests mostly feel like config tests. I'm going to leave everything in convert_markdown for now and wait for validate_config to be improved generally.


volto-slate does not properly support "list in list" as generated by Plone Classic

In Plone classic, create a list in a list in a richtext field. This is how do to it:

create a list with a single line
create another line in the list (another list item)
increase the level of the list item

The generated HTML is:

blabla

dasdasdas

Notice that the sublist is a part of the list item.
If you try the same thing with Volto (you can increate the level of the elements using the Tab key at the beginning of the list item), you'll get the following markup:

blabla

dasdasdas

If you copy the rendered markup of the Plone Classic above list and paste it in a Volto block, it will "silently eat" the sublist. If you generate the proper slate structure (using code from the plone.volto slate support PR), everything will appear to be working, but when you try to do any editing on the text block, the sublist will be swallowed.

Here's a page with sublists as a sample: https://climate-adapt.eea.europa.eu/en/eu-adaptation-policy/strategy

this is "expected behavior" because config.slate.inlineElements does not contain either ul or li elements.
According to the TinyMCE generated markup, they should be inline.

@tiberiuichim What do you mean that they should be inline according to the TinyMCE markup? The TinyMCE markup for nested lists is correct (there is a similar example at https://developer.mozilla.org/en-US/docs/Web/HTML/Element/ul) but lists and list items are block-level elements in HTML, not inline -- https://developer.mozilla.org/en-US/docs/Web/HTML/Block-level_elements

@davisagli I agree, it's just that I need to understand how to align the expectations of slate vs our internal model.
The problem we're hitting one of these two conditions: https://github.com/ianstormtaylor/slate/blob/1d8010be8500ef5c98dbd4179354db1986d5c8f4/packages/slate/src/create-editor.ts#L236-L237C13
Basically, for an element
text
...

, the first child (node.children[0]) is an inline element (Text), so slate expects that all the other children are also inline elements.
So, in my screenshot, when Slate is normalizing the parent
 and it reaches the
 child, isInlineOrText is False for the ul, so it just removes it.

This comment from slate code is a bit head-scratching:
https://github.com/ianstormtaylor/slate/blob/1d8010be8500ef5c98dbd4179354db1986d5c8f4/packages/slate/src/create-editor.ts#L253-L256
        // Only allow block nodes in the top-level children and parent blocks
        // that only contain block nodes. Similarly, only allow inline nodes in
        // other inline nodes, or parent blocks that only contain inlines and
        // text.

@tiberiuichim I think it means we have to turn

  some text

 ...

into

some text

 ...

so that the text is not parented by a node that contains both block-level and inline elements.

@davisagli as an additional piece of information, volto-slate uses almost exclusively Element nodes (blocks), but declares them to be "inline", for example https://github.com/plone/volto/blob/master/packages/volto-slate/src/editor/plugins/Link/extensions.js#L11


Consider a workflow for integrating the db artifact w/ a save & in-memory workflows..

Seems too bad if we would have to decrypt the db and drop it into user:// in order to interact with it. I haven't looked at the sqlite3 C API.. can you interact with a db entirely in memory? This would allow you to pull the db blob from an encrpyted save file, load it into memory and work with it that way, so that you don't have to drop the db into user:// where it can be yoinked/messed-with by users. Obviously we're just talking about security-through-obscurity and raising the bar a bit, here. It's not perfect, but I don't want to deliver all the persistable game data to the users on a silver platter.
I really like this library and have started using it (having a queryable DB is great when, you know, you need performance for pulling from large datasets heh). This is something that's been on my mind and I would like to be able to address before it ships.
I haven't taken any time to look at the source, but I guess I wanted to let you know that at least one other person is using this library and is thinking about these problems :)

About the database in memory, isn't SQLite supporting this out-of-the-box by setting the path to :memory: (sqlite/sqlite3.h#L2819) ?
Maybe you could give it a try.

o.O

@khairul169, is that a yes? :laughing:

Yeah, i think :blush:
Don't know what is memory overflow for, but :memory: is likely does what he want :+1:
For security purpose, maybe you can save data to encrypted file using godot's filesystem.. Then decrypt, load & insert them to sqlite db loaded with :memory: path. But i don't know how to save sqlite data modified in the memory to encrypt file.. There is no such as dump function for sqlite :sob:

I think a better alternative is to use the VFS layer in SQLite to wrap FileAccessEncrypted I/O class already provided by Godot.


Enable support for keccak again

This PR drops pysha3 as dependency and adds support for keccak again

Excellent work, thank you!

Thanks for merging it. Could we get a release with the fix?

Done, multiformats>=0.3.1.post4 has now been released with the fix.


New-DbaDbTable - ensure schema is created if it doesn't exist

Type of Change

[ ] Bug fix (non-breaking change, fixes # )
[x] New feature (non-breaking change, adds functionality, fixes #7153 )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[x] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[x] Adding code coverage to existing functionality
[x] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose
Ensure that the schema is created instead of returning an error to the caller.
Approach
Minor change to create schema before creating the table.
Commands to test
See the new integration tests.

@jpomfret, just a quick note that I'll need to update this PR after #7178 is merged.


lets avoid using the image name docker property by default (we could maybe enable it with a setting later?)

lets avoid using the image name docker property by default (we could maybe enable it with a setting later?)

[merge]


Escape arguments by default

Since that matches std::process::Command more closely.
Breaks with weird remote shells, but raw_ provides an escape hatch.
https://github.com/sfackler/shell-escape/issues/5 is a little sad.

Released on 0.6.0.


TST: refactor to use sphinx test fixtures

Based on what @chrisjsewell did in https://github.com/executablebooks/sphinx-book-theme/pull/211 (https://github.com/executablebooks/sphinx-book-theme/pull/211/commits/1a55f279f519b14ee2fb629ce308af21fa58da1e)

I am going to merge this, as I need it for other PRs where I want to properly add tests (the current implementation doesn't work for multiple "sites"). I think it should be uncontroversial since it more or less a straight copy from executablebooks ;)

I am going to merge this, as I need it for other PRs where I want to properly add tests (the current implementation doesn't work for multiple "sites"). I think it should be uncontroversial since it more or less a straight copy from executablebooks ;)

Looks good to me 😀

Looks good to me 😀


Add pre-discharge support

I got the switches working via MQTT but found my JK_Bms also having a Pre-charge switch. This is a FET with a resistor in parallel over the discharge FET, the purpose of this is to charge capacitors in your load before switching on the discharge port and create a high current peak. It would be great to be able to have the possibility to switch on this pre-discharge circuit over MQTT
My BMS is:
JK_BD4A24S4P
HW Ver. V11.XW
SOFT Ver.11.38

The command isn't supported yet. Do you own an Android device and could you prepare a btsnoop capture of the BLE traffic while turning the featre on/off?

My BMS
[13:13:28][I][jk_bms_ble:1223]:   Vendor ID: JK_BD6A17S6P
[13:13:28][I][jk_bms_ble:1224]:   Hardware version: 11.XW
[13:13:28][I][jk_bms_ble:1225]:   Software version: 11.27

doesn't show this feature at the Android app. Could you provide a screenshot how it looks like on your site?

Dischrg. Pre. Chrg. T(S) setting
aa:55:90:eb: 25 :04: 04:00:00:00 :36:d5:24:4a:68:8e:ad:95:5f:b7 # 0x04 = 4S
aa:55:90:eb: 25 :04: 05:00:00:00 :3c:35:b5:c4:8c:dd:6c:11:32:aa # 0x05 = 5S


Fix typehint

Q
A

Is bugfix?
❌

New feature?
❌

Breaks BC?
❌

Tests pass?
❌

Widget::begin(), Widget::end(), Widget::widget() at now has typehint for child class

Thanks!


IntelliSense errors in ASP.NET Core 2.0, targeting full .NET Framework

I'm getting IntelliSense errors while attempting to author an ASP.NET Core 2.0 application on Windows, targeting the full .NET Framework (not .NET Core).  IntelliSense tells me I need to add a reference to netstandard, Version=<IP_ADDRESS>, which seems odd, as .NET Standard is an "interface" not an implementation.  If anything I'd expect it to instruct me to add a reference to mscorlib or something. The project builds and runs just fine from the command line.
I've uploaded a repro project here, which includes screenshots:  https://github.com/dsteinweg/vscode-netcore2
Hopefully this is the right place for this issue, and hopefully I'm not just missing something simple.  Thanks in advance, loving the full ecosystem of .NET Core tooling!
Environment data
dotnet --info output:
.NET Command Line Tools (2.0.0)

Product Information:
 Version:            2.0.0
 Commit SHA-1 hash:  cdcd1928c9

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 RID:         win7-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.0.0\

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0
  Build    : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d

VS Code version: 1.15.1 41abd21afdf7424c89319ee7cb0445cc6f376959, 64-bit
C# Extension version: 1.12.1
Steps to reproduce

Create a new ASP.NET Core 2.0 project on Windows with dotnet new web
Change the TargetFramework in .csproj to net461 or net47
Remove Microsoft.AspNetCore.All PackageReference (this new meta-package only seems to apply if you're targeting .NET Core)
Add PackageReferences for Microsoft.AspNetCore and Microsoft.AspNetCore.Mvc, version 2.0.0

Expected  behavior
VS Code IntelliSense works as you'd expect, with hover support, "dotted" completion, etc.
Actual behavior
VS Code IntelliSense shows red squiggly errors, mentioning that you must add a reference to netstandard, Version=<IP_ADDRESS>.

Thanks for the clear repro steps. I can definitely reproduce this issue. Taking a look...

Hmmm... I was able to repro it once but couldn't repro it again after performing a dotnet restore from the command line.

Ah, I see the difference. If I load it from a VS 2017 Developer Command Prompt, it works fine because MSBuild loads its toolset from VS 2017.

Could it be that the version of MSBuild.exe that is installed with OmniSharp doesn't support .NET Core/Standard 2.0..?
I see this in the OmniSharp Log:
[info]: OmniSharp.MSBuild.MSBuildProjectSystem
        Omnisharp will use local MSBuild.
    MSBUILD_EXE_PATH: C:\Users\dsteinweg\.vscode\extensions\ms-vscode.csharp-1.12.1\.omnisharp\msbuild\MSBuild.exe
    MSBuildExtensionsPath: C:\Users\dsteinweg\.vscode\extensions\ms-vscode.csharp-1.12.1\.omnisharp\msbuild

Sadly I can't execute MSBuild.exe at that location to call --version or equivalent [0], but the Details tab from right-click Properties shows File version of 15.3.388.41745, Product version 15.3.388-preview+g11a3c31abc.
By contrast, MSBuild.exe in the VS2017 15.3 installation folder at C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin shows File version 15.3.409.57025, Product version 15.3.409+gc1de301405, which seems to be a higher version than OmniSharp's MSBuild.
[0]:
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build.Framework, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)
   at Microsoft.Build.CommandLine.MSBuildApp.Main()

OmniSharp contains a bare-bones MSBuild that's a bit older. I'm updating that to see if it addresses the problem.
Normally, OmniSharp will try to locate VS 2017 on your box and use the MSBuild there if possible. However, it appears that isn't working. Could you try launching VS Code form your VS 2017 Developer Command Prompt? That should give OmniSharp the context it needs to locate the MSBuild within VS 2017.

Aha!  That works!  I see this in the OmniSharp log now, it's using VS2017's MSBuild instead of OmniSharp's:
[info]: OmniSharp.MSBuild.MSBuildProjectSystem
        OmniSharp initialized with Visual Studio MSBuild.

And those red squigglies are gone.
Could it simply be that VS2017's MSBuild.exe isn't in my PATH?

Hmm, adding C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin to my path didn't cause OmniSharp to find VS2017's MSBuild.exe.
Maybe setting the MSBuildSdksPath in omnisharp.json, as you indicated here...

Hmm, didn't seem to help.  I tried 3 different values for MSBuildSdksPath in omnisharp.json at the root of my project, they all resulted in this error [0]:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Sdks

[0]:
Microsoft.Build.Exceptions.InvalidProjectFileException: The SDK 'Microsoft.NET.Sdk.Web' specified could not be found.  c:\github\vscode-netcore2\vscode-netcore2.csproj
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImportsFromUnescapedImportExpressionConditioned(String directoryOfImportingFile, ProjectImportElement importElement, List`1& projects, Boolean throwOnFileNotExistsError)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImports(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate(ILoggingService loggingService, BuildEventContext buildEventContext)
   at Microsoft.Build.Evaluation.Project.Reevaluate(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.Project.ReevaluateIfNecessary(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.Project.Initialize(IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.Project..ctor(String projectFile, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.ProjectCollection.LoadProject(String fileName, IDictionary`2 globalProperties, String toolsVersion)
   at OmniSharp.MSBuild.ProjectFile.ProjectFileInfo.LoadProject(String filePath, String solutionDirectory, String sdksPath, ILogger logger, MSBuildOptions options, ICollection`1 diagnostics, ImmutableArray`1& targetFrameworks)
   at OmniSharp.MSBuild.ProjectFile.ProjectFileInfo.Create(String filePath, String solutionDirectory, String sdksPath, ILogger logger, MSBuildOptions options, ICollection`1 diagnostics)
   at OmniSharp.MSBuild.MSBuildProjectSystem.LoadProject(String projectFilePath)

Adding folders to your path wouldn't do the trick. Instead, you can set the following environment variables that MSBuild uses from the VS Developer Command Prompt:

Set VSINSTALLDIR to your VS installation directory. From your comments above, it looks like that should be: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise
Set VisualStudioVersion to 15.0

FWIW, I wouldn't recommend setting MSBuildSdksPath as I mentioned in that other thread. In this case, the person who filed the issue was using a highly customized set up.

This is just a workaround. I've discovered what the issue is, but need to spend some time considering what the best fix would be.

Setting those two environment variables works!  That's a better workaround than launching Code from the Developer Command Prompt every time.  Thanks for the sleuthing Dustin, I'll be able to continue working in VS Code for now (yay!).
Since you were able to reproduce it initially on your machine, I take it that means it's not a machine-specific (mis-)configuration on our machines, and that there is an underlying problem somewhere?
Also, I've only been testing this on my Win7 work machine thus far, but I can try from my personal machine tonight (Ubuntu 16.04) to see if the same problem exists.  Of course, one can't install VS2017 on Linux, so I don't imagine this environment variable workaround would work.

Nope this isn't machine specific. It's related to a bunch of complexity added that allows .NET Framework TFMs to target netstandard2.0. I'm trying to figure out the best way to include this in OmniSharp.
No, the environment variable trick wouldn't work on Linux. It might (with many caveats) work if you have Mono 5.2.0 or later installed, but I haven't verified that.

Any workarounds that don't rely on VS2017?

I was experiencing the same issue and starting VS Code via the Developer Command Promt is working for me. But I've installed VS2017 and setting the environment variables doesn't do the trick for me.
SET VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\
SET VisualStudioVersion=15.0

Are there any additional steps or things to consider when having installed multiple versions of Visual Studio?

There have been lots of improvements since this issue was opened. Please open a new issue with logs if you are still running into this.


Confused about the anchor settings in fpn neck when using nuScenes dataset

I'm confused about the anchor settings in fpn neck when using nuScenes dataset, could you tell me why setting anchor sizes like this?
https://github.com/open-mmlab/mmdetection3d/blob/440e44a0e03d575bf682d35ab2771e88f774f4dc/configs/_base_/models/hv_pointpillars_fpn_nus.py#L51-L56

This is an empirical setting and may not be the best hyper-parameters.
As the scale of objects ranges from 0.5x0.5 to 4 x 8，we design the anchor to have different aspect ratios and scales to cover all the objects. Thus, in the last pyramid level, the biggest anchor will be [0.86604, 2.59814, 1.*4].


Split IBitcoindRpc into separate server-side and client-side traits

IBitcoindRpc contains methods that aren't supposed to be available on the fedimint servers such as get_txout_proof. Consider splitting this into separate traits for client and server.

If someone takes this on and has trouble understanding the Dyn* pattern ping me or @dpc.

For the RBF UTXO management we might need get_txout_proof for last few blocks (pruned node) for server side. @bradleystachurski , not sure if there's an issue for that already.

@dpc Nope, no issue yet. I'm planning on creating issues for the RBF work once everyone has a chance to review the proposal and agrees it's a sane approach.
If nobody else grabs this, I'll pick it up. For visibility, I won't have the bandwidth until early November but if nobody else has started working on this I'll self assign.

Hey @bradleystachurski , are you still working on it?

Hey @15IITian, thanks for checking! I'm not currently working on this so feel free to grab it.

@15IITian The initial issue description is clear. If that doesn't make sense, this probably isn't a good issue for you to work on. I'm removing the "good first issue" label because this requires some context about Fedimint architecture.

OK


(Re)Write test cases for minio-java

Current minio-java has functional tests but we need to take a fresh look at the tests -
Tests should

Complete fast. No stress tests.
Be named to indicate the use case they are testing.
Access all the public Minio server APIs via minio-java.
Include positive case and edge cases for each API.
Refer test files from a static data directory as in (https://github.com/minio/minio-go/pull/727/files#diff-81a03de42b955e14be71afa81768a78cR52). This helps keep the Mint container light weight and avoids the file creation in runtime, making the tests run faster.

Tests should not include

Big file uploads.
Multiple tests related to the same API.

Closing as fixed by https://github.com/minio/minio-java/pull/586


Writing Custom Adversarial Loss Function

Is it possible to incorporate the output from a discriminator function in Keras?
For example is it at all possible to create something to this effect?
def adversarial_loss(y_true, y_pred):     return K.log(1-discriminator.predict(y_pred))
where 'discriminator' refers to a model that takes in y_pred and returns a 0 to 1 value.
It appears there have been Tensorflow implementations which combine contextual and perceptual loss:
https://bamos.github.io/2016/08/09/deep-completion/
Any help is greatly appreciated.

Yes this is definitely possible:
def create_adv_loss(discriminator):
    def loss(y_true, y_pred):
        return K.log(1.0 - discriminator.predict(y_pred))
    return loss

discriminator = Model(...)
model = Model(...)
model.compile(..., loss=create_adversarial_loss(discriminator))

Thanks for the input.
Have you been able to get this kind of implementation working? I run into issues as 'y_pred' is treated like a Tensor Variable, not an array suitable for passing as a model input.

Is there any working code for using predict function in custom loss? Any help is appreciated

a frozen discriminator is the same as a loss function.
May I ask why these 2 are the same? I just don't get it. The D and G ought to have differenet loss functions.

@kevinwu23

The output of a discriminator function with BSE loss can be used as your adversarial loss. Keras just doesn't allow for this explicit formulation.


[Doc Update][ME plugins update]

updated

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/messaging-extensions/build-bot-based-plugin.md
94
27
link
:white_check_mark:

msteams-platform/messaging-extensions/high-quality-message-extension.md
93
32
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.


Add some unit test for utils, storage, rafs, api and builder.

Relevant Issue (if applicable)
If there are Issues related to this PullRequest, please list it.
Details
Please describe the details of PullRequest.
Types of changes
What types of changes does your PullRequest introduce? Put an x in all the boxes that apply:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation Update (if none of the other choices apply)

Checklist
Go over all the following points, and put an x in all the boxes that apply.

[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.

Great work, thanks! @ZouTaooo
cc @jiangliu @adamqqqplay Let's take a look together.

@ZouTaooo Thanks for your contribution! It looks really good.


Include 1.10 in testing matrix and update other django versions

Coverage decreased (-0.4%) to 94.032% when pulling 576c28968930b05e711a9c856eb6cd60b5cab21c on orf:patch-1 into 08d49e02fae735ed8befbe56defc30ee34ce889d on kencochrane:master.

Coverage decreased (-0.4%) to 94.032% when pulling 576c28968930b05e711a9c856eb6cd60b5cab21c on orf:patch-1 into 08d49e02fae735ed8befbe56defc30ee34ce889d on kencochrane:master.

Coverage decreased (-0.4%) to 94.032% when pulling 48408de75bfe2fd7092d6146de780857b8aa4c43 on orf:patch-1 into 08d49e02fae735ed8befbe56defc30ee34ce889d on kencochrane:master.

Coverage decreased (-0.4%) to 94.032% when pulling 48408de75bfe2fd7092d6146de780857b8aa4c43 on orf:patch-1 into 08d49e02fae735ed8befbe56defc30ee34ce889d on kencochrane:master.

Looks like the Travis builds failed with current config

closing since this is out of date, and handled by other PRs


Update MIDI device list when MIDI device connected/disconnected

At the top right of the app, a list of MIDI devices is displayed.

However, it does not update automatically (requires refreshing the page). Please help to make list of MIDI devices update on-the-fly.

Hey, I'd like to work on this!

This is already done.


User Details Fails

I'm getting a 500 error in step details of users

What is the actual error text?
Did you setup the Oauth Plugin and create an app?
I tested this on the latest of everything and it works fine.

I am getting this error
Received HTTP status code [400] with message "No OAuth parameters supplied" when getting temporary credentials.
everything being followed with the last updates , any help !?


Implementation of real-time object detecion

Hi, giuliogamba,
Really thanks for your QNN tutorial ,recently, i want to achieve real-time object detection by QNN with pynq, i changed the tiny-yolo-image-loop.ipynq code, capture the image through usb webcam in the loop because i want to achieve real-time detection in this way, but my frame rate is only 1/4 FPS, that means it takes 4 seconds to load an image, it is too slow, no real-time, so i want to ask how can i improve my frame rate? I am new to pynq domain, hope get some advise from you.
Hope to hearing from you soon, and thanks in advance!
Icarus.

I would suggest you to read this paper.
In there we explain how we were able to achieve 18 fps on the Ultra96 board. The hardware used is the same you can find in this project, but we haven't released the integration with darknet.
Hope that work will provide insight on how to achieve what you want.

What is the expected fps for an optimized pynqz1-z2?


Adds setting to enable/disable Thread Contention Monitoring

Signed-off-by: Surya Sashank Nistala<EMAIL_ADDRESS>Is your feature request related to a problem? Please provide an existing Issue # , or describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you are proposing
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #171 (1dfdae8) into main (0f55dd5) will increase coverage by 0.20%.
The diff coverage is 75.86%.

:exclamation: Current head 1dfdae8 differs from pull request most recent head 24cce02. Consider uploading reports for the commit 24cce02 to get more accurate results

@@             Coverage Diff              @@
##               main     #171      +/-   ##
============================================
+ Coverage     72.29%   72.50%   +0.20%
- Complexity      356      369      +13
============================================
  Files            44       44
  Lines          2516     2575      +59
  Branches        168      176       +8
============================================
+ Hits           1819     1867      +48
- Misses          591      597       +6
- Partials        106      111       +5

Impacted Files
Coverage Δ

...dler/PerformanceAnalyzerClusterSettingHandler.java
73.58% <59.09%> (+0.32%)
:arrow_up:

...analyzer/config/PerformanceAnalyzerController.java
78.08% <72.22%> (-1.12%)
:arrow_down:

...ig/setting/PerformanceAnalyzerClusterSettings.java
92.30% <100.00%> (+0.64%)
:arrow_up:

...config/PerformanceAnalyzerClusterConfigAction.java
88.67% <100.00%> (+1.44%)
:arrow_up:

...action/config/PerformanceAnalyzerConfigAction.java
93.18% <100.00%> (+0.97%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f55dd5...24cce02. Read the comment docs.


Parsing annotation field as a promQL expression

Hey there, this tool looks very interesting and it has the potential to solve some issues that we've been experiencing as of late.
In our use-case, we have the need to embed extra promQL expressions in alert annotations for use by other external systems to query against Prometheus. We have the need to validate that these expressions when running builds, but as you know, promtool only checks the promQL validity of the expr field, and doesn't check other fields like annotations.meta_query.
Would you be interested in including a feature in the project like the above as an additional type of rule check? Myself or one of my colleagues would be willing to contribute if it's something you'd consider including in the project.
====
To explain more of our use-case with examples, here is an example alert definition that we have:
   - alert: WindowsNodeDiskWillFillIn4Hours
        expr: predict_linear(wmi_logical_disk_free_bytes[2h], 4 * 3600) < 0
        for: 5m
        labels:
          env: prd
        annotations:
          summary: "Node {{$labels.instance}} is predicted to fill its disk in less than 4 hours."
          meta_query: 'count(label_replace(wmi_logical_disk_free_bytes, "host", "$1", "instance", "(.*)")) by(host)'

You can see that the meta_query is a little different from the expr - it's basically for the purpose of a different non-prometheus system 'discovering' what servers are actually exposing this metric to prometheus.
We have the need to ensure that meta_query is also a valid promQL expression - i.e. not missing parenthesis, etc.

Hi! I'm glad you find it useful.
Interesting use case with PromQL in the annotation. It should be quite easy, so I do not see reason to not add it if there is demand for it. :)
The unfortunate thing is you cannot write tests for those queries. In theory, you could use recording rules for those but I get that having a recording rule for everything if a bit annoying.

@neufeldtech PTAL if this would resolve your issue https://github.com/FUSAKLA/promruval/pull/3
It just tries to parse the annotation contents as a PromQL if present.
Other validation rules are not checked on the PromQL expression in annotation such as data limits, minimum range vector selectors, usage of labels etc since that would require the validation checks to be aware of each other and that would require bigger refactoring.

@FUSAKLA yep, all that I need is to check for basic syntax so this would work!
Curious though why you're using the influxdata promQL package instead of the Prometheus one?

That is off course mistake :facepalm:
Wondering when and how that happened. Already foxed bask to official PromQL parser package.
Thanks for pointing that out!

@neufeldtech version 1.2.0 should be out now with the new annotationIsValidPromQL.
Please let me know if that works out for you


[BUILD] Some fixes to make Ratex compatible with PyTorch v1.12.

Description
This PR addresses points 3 and 6 in issue #6. Also includes some minor changes to QUICKSTART.md.
Checklist

[x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage
[x] Code is well-documented

cc @awslabs/raf-reviewer

Override CI result and merge.
Thanks @zhouyuan1119


Support configuration revision

Configuration revision is not changed when subscribing so I think we can leave this to the publisher plugin


[WIP] Add other OS builds

[ ] OSX
[ ] Windows + conda

Codecov Report

Merging #17 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #17   +/-   ##
=======================================
  Coverage   80.61%   80.61%
=======================================
  Files           4        4
  Lines          98       98
=======================================
  Hits           79       79
  Misses         19       19

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 581b9b8...bac82f2. Read the comment docs.


Add content for Someone I know is Sick

Add content for https://www.coronasafe.in/action/someone-i-know-is-infected
Primary source for consultation : https://www.cdc.gov/coronavirus/2019-ncov/hcp/guidance-prevent-spread.html#precautions

#6 fixes this


file=http:// works, not file=https://

In Chrome & Apache web server and secure layer https in Debian GNU/Linux 10 (buster)
OK with
http://www.quellidel68.it/pdfview/web/viewer.html?file=http://www.quellidel68.it/doc/1968/151168studentigreci.pdf
Note: file=http://
KO with
http://www.quellidel68.it/pdfview/web/viewer.html?file=https://www.quellidel68.it/doc/1968/151168studentigreci.pdf
Note: file=https://
https://www.quellidel68.it/doc/1968/151168studentigreci.pdf has not problems, and is readable in Chrome
pdfjs-2.7.570-dist
Chrome  90.0.4430.212
Furio

The error message is pretty self-explanatory: "file origin does not match viewer's". In the second link the file origin is HTTPS whereas the viewer origin is HTTP, so that's a mismatch. The second link does work it the viewer also uses HTTPS: https://www.quellidel68.it/pdfview/web/viewer.html?file=https://www.quellidel68.it/doc/1968/151168studentigreci.pdf
Closing since this is not a bug.


task: Subsume keys to check

This collapses the keys seen. Removing keys that have been subsumed by a wider key (a key that includes the same projects or more as existing keys).

One thing. Quite a bit of this logic seems like it's the same no matter where we store things, so if anyone wants to have a once-over on our abstraction layer I think that could simplify adding new datasources.


wget exiting with error - not creating folder?

magzdb version: 1.1.2
Python version: 3.7.3
Operating System: Debian 10.5

Description
When running with --downloader wget, magzdb exits with error. It seems to be that wget isn't creating the correct folders.
What I Did
I ran the following,
user01@s107487:~/books/temp_magazines$ /home/user01/.local/bin/magzdb -i 1593 -e 4056530 --downloader wget --debug
2020-09-25 06:01:11.903 | INFO     | magzdb.magzdb:download:199 - Found 1 editions of Everyday_practical_electronics
2020-09-25 06:01:11.903 | INFO     | magzdb.magzdb:download:204 - Downloading year 2020 issue 8
2020-09-25 06:01:11.904 | DEBUG    | magzdb.magzdb:_print:52 - Issue ID: 4056530
2020-09-25 06:01:11.988 | DEBUG    | magzdb.magzdb:_print:52 - Download Link ID: 675386
2020-09-25 06:01:12.084 | DEBUG    | magzdb.magzdb:_print:52 - Download URL: http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf
2020-09-25 06:01:12.085 | DEBUG    | magzdb.magzdb:_print:52 - wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"
Traceback (most recent call last):
  File "/home/user01/.local/bin/magzdb", line 10, in <module>
    sys.exit(main())
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/cli.py", line 96, in main
    filter=args.filter,
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/magzdb.py", line 241, in download
    subprocess.run(command)
  File "/usr/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"': 'wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"'

If I first run the command without --downloader, as expected the file downloads and I get...
user01@s107487:~/books/temp_magazines$ /home/user01/.local/bin/magzdb -i 1593 -e 4056530 --debug
2020-09-25 06:50:37.830 | WARNING  | magzdb.cli:main:81 - Use of external downloader like wget or aria2 is recommended
2020-09-25 06:50:38.245 | INFO     | magzdb.magzdb:download:199 - Found 1 editions of Everyday_practical_electronics
2020-09-25 06:50:38.245 | INFO     | magzdb.magzdb:download:204 - Downloading year 2020 issue 8
2020-09-25 06:50:38.245 | DEBUG    | magzdb.magzdb:_print:52 - Issue ID: 4056530
2020-09-25 06:50:38.328 | DEBUG    | magzdb.magzdb:_print:52 - Download Link ID: 675386
2020-09-25 06:50:38.448 | DEBUG    | magzdb.magzdb:_print:52 - Download URL: http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf
2020-09-25 06:50:38.926 | DEBUG    | magzdb.magzdb:_print:52 - Downloading to /home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf

Without removing the file/folder created in the last step, If I then run the wget command that DEBUG printed in the first attempt,
user01@s107487:~/books/temp_magazines$ wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"
--2020-09-25 06:51:41--  http://file.magzdb.org/ul/1593/Practical%20Electronics%20-%202020-08.pdf
Resolving file.magzdb.org (file.magzdb.org)... <IP_ADDRESS>
Connecting to file.magzdb.org (file.magzdb.org)|<IP_ADDRESS>|:80... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

Again, without deleting the file/folder created and running the first command again, it exits with the same error,
user01@s107487:~/books/temp_magazines$ /home/user01/.local/bin/magzdb -i 1593 -e 4056530 --downloader wget --debug
2020-09-25 06:55:39.899 | INFO     | magzdb.magzdb:download:199 - Found 1 editions of Everyday_practical_electronics
2020-09-25 06:55:39.900 | INFO     | magzdb.magzdb:download:204 - Downloading year 2020 issue 8
2020-09-25 06:55:39.900 | DEBUG    | magzdb.magzdb:_print:52 - Issue ID: 4056530
2020-09-25 06:55:39.976 | DEBUG    | magzdb.magzdb:_print:52 - Download Link ID: 675386
2020-09-25 06:55:40.090 | DEBUG    | magzdb.magzdb:_print:52 - Download URL: http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf
2020-09-25 06:55:40.090 | DEBUG    | magzdb.magzdb:_print:52 - wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"
Traceback (most recent call last):
  File "/home/user01/.local/bin/magzdb", line 10, in <module>
    sys.exit(main())
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/cli.py", line 96, in main
    filter=args.filter,
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/magzdb.py", line 241, in download
    subprocess.run(command)
  File "/usr/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"': 'wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"'

@Rob-T, I think the issue is related to wget, since wget would not create directory when downloading.
In your case directory Everyday_practical_electronics needs to exist.
To resolve this issue you can create the Everyday_practical_electronics or use aria2 downloader until the fix is released.

@Rob-T I've fixed this bug in the latest release v1.1.3

I've updated to v1.1.3 and also installed aria2, but am still having issues (with both wget and aria2 actually).
With wget...
user01@s107487:~/books/temp_magazines$ /home/user01/.local/bin/magzdb -i 1593 -e 4056530 --downloader wget --debug
2020-09-25 08:55:56.433 | INFO     | magzdb.magzdb:download:202 - Found 1 editions of Everyday_practical_electronics
2020-09-25 08:55:56.433 | INFO     | magzdb.magzdb:download:207 - Downloading year 2020 issue 8
2020-09-25 08:55:56.433 | DEBUG    | magzdb.magzdb:_print:52 - Issue ID: 4056530
2020-09-25 08:55:56.606 | DEBUG    | magzdb.magzdb:_print:52 - Download Link ID: 675386
2020-09-25 08:55:56.699 | DEBUG    | magzdb.magzdb:_print:52 - Download URL: http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf
2020-09-25 08:55:56.699 | DEBUG    | magzdb.magzdb:_print:52 - wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"
Traceback (most recent call last):
  File "/home/user01/.local/bin/magzdb", line 10, in <module>
    sys.exit(main())
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/cli.py", line 96, in main
    filter=args.filter,
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/magzdb.py", line 244, in download
    subprocess.run(command)
  File "/usr/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"': 'wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"'

With Aria2...
user01@s107487:~/books/temp_magazines$ /home/user01/.local/bin/magzdb -i 1593 -e 4056530 --downloader aria2 --debug
2020-09-25 08:56:29.388 | INFO     | magzdb.magzdb:download:202 - Found 1 editions of Everyday_practical_electronics
2020-09-25 08:56:29.388 | INFO     | magzdb.magzdb:download:207 - Downloading year 2020 issue 8
2020-09-25 08:56:29.388 | DEBUG    | magzdb.magzdb:_print:52 - Issue ID: 4056530
2020-09-25 08:56:29.465 | DEBUG    | magzdb.magzdb:_print:52 - Download Link ID: 675386
2020-09-25 08:56:29.534 | DEBUG    | magzdb.magzdb:_print:52 - Download URL: http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf
2020-09-25 08:56:29.534 | DEBUG    | magzdb.magzdb:_print:52 - aria2c -c --dir="/home/user01/books/temp_magazines/Everyday_practical_electronics" --out="Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"
Traceback (most recent call last):
  File "/home/user01/.local/bin/magzdb", line 10, in <module>
    sys.exit(main())
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/cli.py", line 96, in main
    filter=args.filter,
  File "/home/user01/.local/lib/python3.7/site-packages/magzdb/magzdb.py", line 244, in download
    subprocess.run(command)
  File "/usr/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'aria2c -c --dir="/home/user01/books/temp_magazines/Everyday_practical_electronics" --out="Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"': 'aria2c -c --dir="/home/user01/books/temp_magazines/Everyday_practical_electronics" --out="Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"'

The folder /home/user01/books/temp_magazines/Everyday_practical_electronics exists when the above was run and currently has the file Practical_Electronics_-_2020-08.pdf in it. If I remove the folder and run it again, the script creates the folder but the download does not happen (the folder is empty).

Can you check if there is a permission issue? Unfortunately I don't have a linux system.

I'm no linux master, but it looks like folders and script both have owner and group as user01, and have drwxr-xr-x attributes (read, write, execute). There doesn't seem to be an issue that I can see. The script has no issues other than when using a downloader.

@Rob-T Can you check if error is resolved by passing shell=True in magzdb/magzdb.py#L244 like this,
- subprocess.run(command)
+ subprocess.run(command, shell=True)

In your case the file is located at "/home/user01/.local/lib/python3.7/site-packages/magzdb/magzdb.py"

Yes, that seems to fix it.  I ran it with both the folder/file existing and not, and the result was a success either way.
user01@s107487:~/books/temp_magazines$ /home/user01/.local/bin/magzdb -i 1593 -e 4056530 --downloader wget --debug
2020-09-25 13:06:01.793 | INFO     | magzdb.magzdb:download:202 - Found 1 editions of Everyday_practical_electronics
2020-09-25 13:06:01.794 | INFO     | magzdb.magzdb:download:207 - Downloading year 2020 issue 8
2020-09-25 13:06:01.794 | DEBUG    | magzdb.magzdb:_print:52 - Issue ID: 4056530
2020-09-25 13:06:01.883 | DEBUG    | magzdb.magzdb:_print:52 - Download Link ID: 675386
2020-09-25 13:06:01.974 | DEBUG    | magzdb.magzdb:_print:52 - Download URL: http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf
2020-09-25 13:06:01.974 | DEBUG    | magzdb.magzdb:_print:52 - wget -c -O "/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf" "http://file.magzdb.org/ul/1593/Practical Electronics - 2020-08.pdf"
--2020-09-25 13:06:01--  http://file.magzdb.org/ul/1593/Practical%20Electronics%20-%202020-08.pdf
Resolving file.magzdb.org (file.magzdb.org)... <IP_ADDRESS>
Connecting to file.magzdb.org (file.magzdb.org)|<IP_ADDRESS>|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16782219 (16M) [application/pdf]
Saving to: ?/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf?

/home/user01/books/temp_magazines/Everyda 100%[====================================================================================>]  16.00M   336KB/s    in 55s

2020-09-25 13:06:57 (295 KB/s) - ?/home/user01/books/temp_magazines/Everyday_practical_electronics/Practical_Electronics_-_2020-08.pdf? saved [16782219/16782219]

Good work!!


add scheduler to WorkerManager and fix http client

Pull Request Test Coverage Report for Build 136

0 of 37 (0.0%)  changed or added relevant lines in 5 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.3%) to 16.336%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

base-node/src/main/scala/com/github/jaitl/crawler/base/worker/config/WorkerConfig.scala
0
1
0.0%

base-node/src/main/scala/com/github/jaitl/crawler/base/worker/WorkerManager.scala
0
4
0.0%

base-node/src/main/scala/com/github/jaitl/crawler/base/worker/WorkerApp.scala
0
7
0.0%

base-node/src/main/scala/com/github/jaitl/crawler/base/worker/http/HttpRequestExecutor.scala
0
8
0.0%

base-node/src/main/scala/com/github/jaitl/crawler/base/worker/http/AsyncHttpRequestExecutor.scala
0
17
0.0%

Files with Coverage Reduction
New Missed Lines
%

base-node/src/main/scala/com/github/jaitl/crawler/base/worker/WorkerApp.scala
1
0.0%

Totals

Change from base Build 130:
-0.3%

Covered Lines:
107

Relevant Lines:
655

💛  - Coveralls


Can't open editor.  Vulkan context error.

Godot version:
f2d8c4eb8
OS/device including version:
Win10 64-bit GeForce 9600 GT/PCIe/SSE2
Issue description:
When trying to open, it immediately closes, with this error.

ERROR: Condition "err" is true. returned: ERR_CANT_CREATE
   at: _initialize_extensions (drivers/vulkan/vulkan_context.cpp:229)
ERROR: Method/Function Failed, returning: m_value
   at: initialize (platform/windows/os_windows.cpp:1470)

I doubt this GPU has any Vulkan support, it's D3D 10 level hardware from ~2008.
But the error message could be improved.

I suspect not either, but is it intended to fallback to OpenGL 2.0 at some point?  Would seem a bit exclusive for those just wanting to make 2D games that aren't very demanding and can run well on old hardware, like in previous versions.

Yes it will fallback to GLES2 if Vulkan is not supported. But GLES2 is not ported yet.

I doubt this GPU has any Vulkan support, it's D3D 10 level hardware from ~2008.

Indeed, Vulkan support is limited to Kepler GPUs (GeForce 600 series) and newer.

Closing as it's a fairly old issue. The error has been improved since then. It's not perfect yet but suggestions to improve it would best be served with a new issue.


CIE Transformation: Why are custom values used

As I understand the Code, the cie function is used to transform the RGB image to grayscale. Why are custom values used and not the default ones, for example provided on the wikipedia page, which would be
0.2126 * R + 0.7152 * G + 0.0722 * B

https://github.com/jwagner/smartcrop.js/blob/11614e14d55d365476bffb5980797d3a76e224cc/smartcrop.js#L562
https://en.wikipedia.org/wiki/Grayscale

Sorry, just seen issue #77 which asks exactly the same question...


Wraps QueryProxy association= and association << in transactions

Solves #766
It was necessary to ensure << happens inside transactions because it supports arrays.

Coverage increased (+0.01%) to 98.43% when pulling cc7720e7ebada3102e39d1a1764c67efb45cb5b8 on transaction_rel_creation into 4a94ea3f841f24214e0d8d0423f7fcf573d7b8bd on master.

Coverage increased (+0.01%) to 98.43% when pulling cc7720e7ebada3102e39d1a1764c67efb45cb5b8 on transaction_rel_creation into 4a94ea3f841f24214e0d8d0423f7fcf573d7b8bd on master.

Coverage increased (+0.01%) to 98.43% when pulling 4ec3a8b1e654950f29f254341d21905cfd422aed on transaction_rel_creation into 4a94ea3f841f24214e0d8d0423f7fcf573d7b8bd on master.

Coverage increased (+0.01%) to 98.43% when pulling ff2b18ee788c437de758c96424cdb0ccec2f13a1 on transaction_rel_creation into 4a94ea3f841f24214e0d8d0423f7fcf573d7b8bd on master.

Coverage increased (+0.01%) to 98.43% when pulling ff2b18ee788c437de758c96424cdb0ccec2f13a1 on transaction_rel_creation into 4a94ea3f841f24214e0d8d0423f7fcf573d7b8bd on master.


Add more warning for service name replacement

The following questions has been repeatedly pop up when a service name replacement is needed, which prevents a successful deployment:

The service name need to be replaced has a port not in "host" mode, meaning it has either port->mode:ingress, or missing port config, we do not warn them if this service name is used in other service that depends on it in any of its environment variables.
An environment variable could be replaced by a config, though we do have an warning, but it is not clear that we do not do service name replacement in the configured values

I rather we fix #396


Make Prometheus labels configurable

@matejv This is now in v2.2.0 if you're using NetBox < 3.5 or in v3.0.0 for NetBox 3.5.


please help change the font size?

First of all, I am using this file very well. Thank you very much!
However, Atlas has made the font size horribly small for my country.
Would you be willing to add a change to the font size of the dialog?
It would save a lot of people in my country if you could.

I also have this problem, especially with prompt UI. I don't think a font-size configuration would be possible, however a UI scale would be more feasible.

I would also love to see an option to either increase font size or UI scale if it's somehow possible. On handhelds like ROG Ally some texts are really small and hard to read.


update testnet deployer scripts

What needs to be done
Update keyutils script which downloads wallet keys from kms and secret manager
Why it needs to be done
The current Testnet deployer machine does not retrieve keys from AWS KMS or the Secret Manager; instead, it manages the keys locally. As a result, these machines are kept permanently.

Updated the deployer scripts on branch : deployment-lisbon. Now we can launch the testnet deployer machine and terminate once deployment completes. All the contract address are stored on aws parameter store and can be fetched during machine setup. To deploy the machine for testnet , use the branch deployment-lisbon

Completed.


[shim] Bad Initialization for vfcntl(), epoll_ctl(), epoll_wait() and epoll_create()

Description
This PR isolates some changes introduced by https://github.com/microsoft/demikernel/pull/1351.

Bypassing this review as it concerns the SHIM Layer and PR https://github.com/microsoft/demikernel/pull/1351.


Telemetry changes

Optimization, Removed Bedrock Id variable as it is not getting accessed anywhere in the working flow

@piyushbhavsar12 can you resolve conflicts in this?

i did, can you please verify


add create customer

Adds create customer action. We will pass a UUID as the name so we have a good primary key to share. They return their own DB key that we need to store. It will get get used when we create a line.

We need a version update?