Are packed files always that huge?

Just a general question that I have. Building the files using yarn build results in the following file sizes:

File
Size
Original Source File Size

main.prod.js
96,1kB
2,314kB

renderer.js
208kB
7,22kB

It's seems odd that the electron main file (the file that loads the BrowserWindow) is actually so much larger than the input file used to create it.
I'm pretty new to the webpack ecosystem, so I would love if someone could help me out here. Why is the file that much larger?
Does it pack everything that mains need (path, electron) and other require calls to the file? And if yes, wouldn't it be more efficient to leave them in node_modules as the directory gets anyway packed into the final electron app by electron-builder?

Here's what I get for the v0.13.3 branch
[0] Hash: 347fa4f7535d866c269f
[0] Version: webpack 4.1.1
[0] Time: 4657ms
[0] Built at: 4/18/2018 5:53:37 PM
[0]                  Asset      Size  Chunks             Chunk Names
[0]     ./app/main.prod.js  93.7 KiB       0  [emitted]  main
[0] ./app/main.prod.js.map   398 KiB       0  [emitted]  main
[0] Entrypoint main = ./app/main.prod.js ./app/main.prod.js.map
[0] [./app/main.dev.js] 3.07 KiB {0} [built]
[0] [./app/menu.js] 5.99 KiB {0} [built]
[0] [assert] external "assert" 42 bytes {0} [built]
[0] [child_process] external "child_process" 42 bytes {0} [built]
[0] [electron] external "electron" 42 bytes {0} [built]
[0] [events] external "events" 42 bytes {0} [built]
[0] [fs] external "fs" 42 bytes {0} [built]
[0] [https] external "https" 42 bytes {0} [built]
[0] [module] external "module" 42 bytes {0} [optional] [built]
[0] [path] external "path" 42 bytes {0} [built]
[0] [util] external "util" 42 bytes {0} [built]
[0]     + 35 hidden modules
[0] npm run build-main exited with code 0
[1] Hash: 3abe981a61ae97609892
[1] Version: webpack 4.1.1
[1] Time: 9523ms
[1] Built at: 4/18/2018 5:53:41 PM
[1]                                  Asset      Size  Chunks                    Chunk Names
[1]   674f50d287a8c48dc19ba404d20fe713.eot   162 KiB          [emitted]
[1]   912ec66d7572ff821749319396470bde.svg   434 KiB          [emitted]  [big]
[1]   b06871f281fee6b241d60582ae9369b9.ttf   162 KiB          [emitted]
[1] af7ae505a9eed503f8b8e6982036873e.woff2  75.4 KiB          [emitted]
[1]  fee66e712a8a08eef5805a46892932ad.woff  95.7 KiB          [emitted]
[1]                       renderer.prod.js   191 KiB       0  [emitted]         main
[1]                              style.css  31.3 KiB       0  [emitted]         main
[1]                   renderer.prod.js.map   557 KiB       0  [emitted]         main
[1]                          style.css.map  86 bytes       0  [emitted]         main
[1] Entrypoint main = renderer.prod.js style.css renderer.prod.js.map style.css.map
[1] [./app/containers/App.js] 1.45 KiB {0} [built]
[1] [./app/containers/CounterPage.js] 1.03 KiB {0} [built]
[1] [./app/containers/HomePage.js] 1.42 KiB {0} [built]
[1] [./app/containers/Root.js] 1.67 KiB {0} [built]
[1] [./app/index.js] 1.93 KiB {0} [built]
[1] [./app/reducers/counter.js] 415 bytes {0} [built]
[1] [./app/reducers/index.js] 548 bytes {0} [built]
[1] [./app/routes.js] 1.84 KiB {0} [built]
[1] [./app/store/configureStore.js] 251 bytes {0} [built]
[1] [./app/store/configureStore.prod.js] 894 bytes {0} [built]
[1] [./node_modules/css-loader/index.js??ref--5-2!./app/app.global.css] ./node_modules/css-loader??ref--5-2!./app/app.global.css 688 bytes [built]
[1] [./node_modules/history/es/index.js] ./node_modules/history/es/index.js + 7 modules 31.9 KiB {0} [built]
[1]        |    8 modules
[1] [./node_modules/react-redux/es/index.js] ./node_modules/react-redux/es/index.js + 14 modules 37.4 KiB {0} [built]
[1]        |    15 modules
[1] [./node_modules/react-router-redux/es/index.js] ./node_modules/react-router-redux/es/index.js + 5 modules 6.51 KiB {0} [built]
[1]        |    6 modules
[1] [./node_modules/redux/es/index.js] ./node_modules/redux/es/index.js + 6 modules 20.6 KiB {0} [built]
[1]        |    7 modules
[1]     + 68 hidden modules
[1]
[1] WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).

Note that webpack bundles include all the libraries that you import so they should definitely be bigger. Some bundles for apps that i've built on the boilerplate are a couple MB's (even up to 13MB in some cases). It really depends on what you're importing and how you import it.

Does Electron load the files still faster compared to declaring every node dependency as external? I imagine that accessing one huge file is faster than multiply small ones.
I was just confused by the difference between the two files.

@SirWindfield it surely does. require() makes recursive reads to the filesystem. Webpack defines imports in a fixed size array so imports inside webpack bundled code is really fast (can assume constant time).
example of what webpack will do if you're importing 3 functions called foo, bar, and baz
[function foo() {}, function bar() {}, function baz() {}, ]


is the multipart/form-data return text like stream ?

Does anyone use this repo? Has it implemented the effect of returning messages letter by letter like stream? Or returning sentence by sentence is also fine.

Stream is not yet implemented see https://github.com/olrea/openai-cpp/issues/19 but it will be.
Yes some people use it.

The stream has to be implemented see https://github.com/olrea/openai-cpp/issues/19. You can use this library for every else without stream though if anything is missing we will add it.


Manpage for the PAM module

Hello,
As requested in #59, here is a manpage for the PAM module (pam_google_authenticator.8).
I also made some modifications to the binary manpage (google-authenticator.1) and in its help message.
Tony

Hand merged.
Thank you!


Bug fix coredump when state combinator input const column

Proposed changes
fix coredump when state combinator input const column

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


[WIP] fix: indexing

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix
[ ] New feature
[ ] Refactor/improvement
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist

[ ] My commit message is conventional
[ ] I have run pre-commit on all files before creating the PR
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] I have checked my code and corrected any misspellings

Codecov Report

Merging #658 (bb5079e) into develop (36f1078) will increase coverage by 0.09%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop     #658      +/-   ##
===========================================
+ Coverage    86.74%   86.83%   +0.09%
===========================================
  Files           25       25
  Lines         1592     1603      +11
===========================================
+ Hits          1381     1392      +11
  Misses         211      211

Impacted Files
Coverage Δ

splunk_connect_for_snmp/snmp/manager.py
85.94% <100.00%> (+0.36%)
:arrow_up:

splunk_connect_for_snmp/splunk/tasks.py
89.39% <100.00%> (+0.24%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:tada: This PR is included in version 1.8.5-beta.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


bump version  0.23.2

Description:
bump  version  0.23.2

Codecov Report
Patch and project coverage have no change.

Comparison is base (788a452) 74.56% compared to head (85af562) 74.56%.

:exclamation: Current head 85af562 differs from pull request most recent head 3ece832. Consider uploading reports for the commit 3ece832 to get more accurate results

Additional details and impacted files
@@              Coverage Diff              @@
##           release/0.23    #1251   +/-   ##
=============================================
  Coverage         74.56%   74.56%
=============================================
  Files                31       31
  Lines              2237     2237
  Branches            445      445
=============================================
  Hits               1668     1668
  Misses              439      439
  Partials            130      130

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


web-app(catalog): clear after search does not work

Description
catalog: have to click to remove focus and then click on x (essentially double click)
Steps to reproduce
1 - write something in search bar in catalog
2 - without clicking on anything else before, attempt to close (press x)

[ ] Verify issue is no longer an issue when #260 gets merged in

issue due to useQueryState re-render, no longer present with #260 changes

Fixed with #260, closing


[Fix] Executing install.sh failed when password and others contain @

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

see issue: https://github.com/apache/dolphinscheduler/issues/6781

Hi @tracehh , to target specific issue from PR, we recommend use keyword fix: #issue_id or close: #issue_id or closes: #issue_id. I would not only connect issue to PR but also close issue automatically when PR is be closed.

Codecov Report

Merging #6782 (2fc8ce3) into dev (088a5a4) will decrease coverage by 0.04%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev    #6782      +/-   ##
============================================
- Coverage     41.86%   41.81%   -0.05%
+ Complexity     3619     3616       -3
============================================
  Files           641      641
  Lines         25897    25897
  Branches       2795     2795
============================================
- Hits          10841    10830      -11
- Misses        14073    14088      +15
+ Partials        983      979       -4

Impacted Files
Coverage Δ

...r/plugin/registry/zookeeper/ZookeeperRegistry.java
47.27% <0.00%> (-7.28%)
:arrow_down:

...org/apache/dolphinscheduler/remote/utils/Host.java
37.77% <0.00%> (-2.23%)
:arrow_down:

...e/dolphinscheduler/remote/NettyRemotingClient.java
52.11% <0.00%> (-1.41%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 088a5a4...2fc8ce3. Read the comment docs.

Hi @tracehh , to target specific issue from PR, we recommend use keyword fix: #issue_id or close: #issue_id or closes: #issue_id in your PR describe. I would not only connect issue to PR but also close issue automatically when PR is be closed.

Hi, @tracehh could you please target issue with specific syntax?

It seem function do not work in my env. I use ]
*OS: mac

Bash: GNU bash, version 5.1.8(1)-release (x86_64-apple-darwin19.6.0)

function charEscape(){
    inputParam=$1
    echo ${inputParam}
    if [[ $1 =~ '@' ]]; then
        tmp=$(echo inputParam | sed "s/@/\\\\@/g")
        inputParam=${tmp}
    fi
    echo "$inputParam"
}

input="abc@123"
input=$(charEscape ${input})
echo ${input}

sed is removed in install.sh file. Another way to config them now? @zhongjiajie

It seem that remove in https://github.com/apache/dolphinscheduler/pull/6840. I going to check what happen to it.

It seem that remove in #6840. I going to check what happen to it.

Hi @tracehh , It seem that this PR should be close, could our remote code delete and we would not use sed to change configure file anymore. Maybe you should found another issue in #5689 or just search issue in our issue list.

when I find out this issue, I think there is a better way to replace 'sed'

As 'sed' removed in install.sh, there are some format style, I change them, please look look @zhongjiajie

@tracehh BTW please change PR title for this PR, and remember close #6781  cause we do not fix this issue anymore.

Thank you all the same. 👍 @tracehh


Updated ReadME for Rust branch

updated ReadME.  Using the PR for a side by side comparison.

This PR updates the ReadME for the Rust branch in preparation for the merge to main.
Other changes include:
-deleting release ntoes
-deleting principles.md
-removing contents from Roadmap.md
In future, I'll submit separate PR's for each of these issues.


Build error when trying migrate to Swift 4

Argument of '#selector' refers to instance method 'displayLinkTick()' that is not exposed to Objective-C
Class affected: CircleProgressView.

Any chance to have the library updated and functional for Xcode 9 and Swift 4? This is a good library and deserves to be updated. Best.

Will be fixed in next build.


u-boot: fix the issue that board can't load the correct dtb

the name like k3-am6528-iot2050-basic-pg2 not the iot2050-basic-pg2.
so it can't find the correnct dtb.
correct this issue
Signed-off-by: chao zeng<EMAIL_ADDRESS>

Let me check the causing commit, and then we will fix that one up. No need for stacking things.

Let me check the causing commit, and then we will fix that one up. No need for stacking things.

this original commit has already upstream,not int the currently patches list.

Unlikely - we added support for PG2 and tested that before upstreaming it.

this is a new bug introduced after the commit 5ac8edaca2fd29c87c2bd4922b464b31a6dbd674.
and this bug is caused by using the fdt-SEQ.

NACK: I've just built the image for PG2 via master and tested it on PG2 Basic. It's all fine. I suspect you accidentally built for PG1.

OK, reproduced now: We better cut off that new prefix which confuses the match. strstr could eventually stumble over "name" vs. "name-suffix". The fix should go into the causing commit (https://github.com/siemens/u-boot/commit/5cf7130b85e9e76ffee765aabb56a972fbe050ed). I'll update the queues.

OK, reproduced now: We better cut off that new prefix which confuses the match. strstr could eventually stumble over "name" vs. "name-suffix". The fix should go into the causing commit (siemens/u-boot@5cf7130). I'll update the queues.

Ok, I  agree. I would drop this PR.


[Bug] Pasting multi-line text

When a multi-line text is pasted to the editor, it only calculates the direction for the first line (node).
Originally reported by @coderyeasin in https://github.com/ueberdosis/tiptap/issues/3085#issuecomment-1505848844

Thank you so much for your response!
when you're done please ensure me a message.
Best of luck!

It should be fixed in the latest version (0.2.3).
I'm gonna close this but feel free to re-open it if you still encounter the issue.


Remove liveness probes

This PR,

upgrades kube-thanos to the latest version
remove liveness probes

cc @squat @brancz @bwplotka @aditya-konarde

Can we increase readiness.. just in case?

@bwplotka Ahh, thanks for reminding that, I was gonna do that! Coming right up!

For readiness probes, we want to simply start probing right away. It doesn't hurt. It can fail so often, but the first time it succeeds we can start serving traffic. Waiting 120s would just result in 120s min downtime for maybe no good reason.

+1 @metalmatze we just need final timeout be 120s


Problem about the code hints

Environment data

Language Server version: 2021.7.2
OS and version: Windows10 21H2
Python version (& distribution if applicable, e.g. Anaconda): 3.9.6

Expected behaviour
See the screenshot below. As I type Class1(, The text marked red should be "Docstring of Class1".
Actual behaviour
The text is actually "Built-in mutable sequence. If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified.", which is actually the docstring of list, i.e. the super class of Class1.
When I hover on Class1, what I see is "Docstring of Class1", which is as expected.
Logs

None

Code Snippet / Additional information

class Class1(list):
    '''
    Docstring of Class1
    '''
    pass
# These empty lines are to avoid the box from occluding the code above.

Class1()  # Please don't just copy & paste this line but type "Class1(" instead.

Dupe of #653 and #1047


Generates twice the connection ID

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
20.6.1
Amplify CLI Version
11.0.5
What operating system are you using?
Mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No
Describe the bug
type User @model @auth(rules: [{ allow: private }]) {
  id: ID!
  name: String
  myOrg: Org @hasOne
  orgs: [Org] @manyToMany(relationName: "OrgUser")
}

type Org
  @model
  @auth(
    rules: [
      { allow: private, operations: [read] }
      { allow: groups, groups: ["Admins"] }
    ]
  ) {
  id: ID! @primaryKey(sortKeyFields: ["userID"])
  name: String!
  userID: ID!
  users: [User] @manyToMany(relationName: "OrgUser")
  user: User @belongsTo(fields: ["userID"])
}

So if I have this models with the relation ships. It creates for the createUserInput following:
id: ID
name: String
userMyOrgId: ID
userMyOrgUserID: ID

This is related both to the myOrg:Org @hasOne relation. If I remove the entire line, then both userMyOrgId and userMyOrgUserID are gone.
As well for the between model which are created automatically has the same issue.
createOrgUser(condition: ModelOrgUserConditionInputinput: CreateOrgUserInput!): OrgUser
CreateOrgUserInput:
id: ID
userId: ID!
orgId: ID!
orguserID: ID!

And I different problem:
So when I add an item in OrgUser. Provide the orgId, orguserID with the orgId and the userId with the userId.
And then query the user I got null.
query MyQuery {
  getUser(id: "someID") {
    orgs {
      items {
        org {
          name
        }
      }
    }
  }
}

If I remove the hasOne relation then it works. But this is a common pattern. That a user as one org because he is the owner and user can be part of multiple orgs and org can have multiple user.
Expected behavior
For createUserInput:
id: ID
name: String
userMyOrgUserID: ID

For CreateOrgUserInput:
id: ID
userId: ID!
orgId: ID!

Reproduction steps
Create the model and push it with amplify push -y
Project Identifier
No response
Log output

# Put your logs below this line

Additional information
No response
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

Hi @juri-diener, My apologies for the delayed response. Could you provide more details on the relationships between 'user' and 'org'? Can a 'user' be associated with multiple 'orgs'?

Hi @juri-diener, I can reproduce the issue by following the details in the description. Hence, marking this as a bug for the team to evaluate further.

Hi @juri-diener, I can reproduce the issue by following the details in the description. Hence, marking this as a bug for the team to evaluate further.

Hi, ok that's good any update on this?


[FIX] #135 - If a timeout is reached, node-nats client should unsubscribe. Or we leak subscriptions.

Changes Unknown when pulling ea6c05849bcbe0f19efbbfa09bb146aa99d15b81 on timeout-should-unsubscribe into ** on master**.

Changes Unknown when pulling 25360cf7b97c8d7f7e681cd51d5746985d67e8cd on timeout-should-unsubscribe into ** on master**.


Script generates invalid download link

Something has changed recently on Zippyshare's side causing the script to generate invalid links.
$ python zippyshare.py
Enter URLs (leave blank to stop): https://www37.zippyshare.com/v/cBO2J0K5/file.html
Enter URLs (leave blank to stop):
[*] Using JSEngine for generating links
[*] 1/1 links parsed https://www37.zippyshare.com/d/123/asd.xml
[*] Verifying download links...
[*] 1/1 links verified
Time taken: 3.223s
[*] All download links saved at links.txt

I am having the same issue

That's because the code has not been updated in like a year and just recently zippyshare added a way to trick downloaders thus they're getting only a asd.xml file, not the actual link.

Even Jdownloader 2 won't work, does anyone have any alternative way at all?

mansuf zippyshare-downloader seems to work

fwiw, I replaced the download url generation part with the one from mansuf's module and got it working again:
import time
import argparse
import requests
import loaders
import logging
from engines.text import TextEngine
from engines.js import JSEngine
from engines.patterns import utils
from concurrent.futures import as_completed, ThreadPoolExecutor
from zippyshare_downloader import extract_info, extract_info_coro

class ZippyParser:
    def __init__(self, workers=10, engine=None):
        self.sess = requests.Session()
        FORMAT = '[*] %(message)s'
        logging.basicConfig(level=logging.INFO, format=FORMAT)
        self.logger = logging.getLogger('Zippyparse')
        self.parser = None
        self.workers = workers
        if engine is None:
            self.engine = JSEngine(logger=self.logger)
        else:
            self.engine = engine(logger=self.logger)
        self.logger.info('Using {} for generating links'.format(self.engine))

    def get_download_link(self, link):
        """
        Parse the contents from the Zippyshare site to extract the actual download link
        of the file. The zippyshare site has javascript logic around how to construct
        the download link.
        """

        # extract, links = self.engine.get_download_link(link)

        file = extract_info(link, download=False)

        return file.download_url, file.url

    def verify_link(self, link):
        """
        Verify that the extracted link points to actual downloadable file.
        In case it points to a Zippyshare site (HTML page), retry the parsing.
        After 3 retries, it gives up.
        """
        count = 0
        while True:
            count += 1
            if count > 8:
                logging.error('{} redirected more than 3 times'.format(link))
                return None

            if not utils.is_valid_link(self.sess, link):
                link = self.get_download_link(link)[0] #utils.get_domain(link)[:-1] + self.get_download_link(link)[0]
                continue

            return link

    def parse_links(self, links):
        """
        Parse the zippyshare page to generate direct download links
        :param links: List of zippyshare URLS
        :return:
        """
        rlinks = []
        failed = []
        count = 0

        with ThreadPoolExecutor(max_workers=10) as executor:
            futures = [executor.submit(self.get_download_link, url) for url in links]
            for f in as_completed(futures):
                extract, link = f.result()
                if extract is None:
                    self.logger.error('Failed to parse - {}'.format(link))
                    failed.append(link)
                    continue
                dlink = extract #utils.get_domain(link)[:-1] + extract
                count += 1
                self.logger.info('{}/{} links parsed {}'.format(count, len(links), dlink))
                rlinks.append(dlink)

        self.logger.info('Verifying download links...')
        flinks = []
        count = 0

        with ThreadPoolExecutor(max_workers=4) as executor:
            futures = [executor.submit(self.verify_link, url) for url in rlinks]
            for f in as_completed(futures):
                if f.result() is not None:
                    flinks.append(f.result())
                    count += 1
                    self.logger.info('{}/{} links verified'.format(count, len(rlinks)))

        return flinks, failed

def load_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--engine', dest='engine', default='js',
                        help='Link generating engine to use for generating links. Valid options are "js" and "text".')
    parser.add_argument('--in-file', dest='infile', default=None, help='path to file containing links to be processed')
    parser.add_argument('--out-file', dest='outfile', default='links.txt',
                        help='path to file in which resultant links will be stored')
    parser.add_argument('--dlc', dest='dlcfile', default=None,
                        help='If you have a dlc file, you can use that instead of a txt file')
    parser.add_argument('--filecrypt', dest='filecrypt_url', default=None,
                        help='Filecrypt link which has a dlc option. Link should not have a captcha or a password.')

    return parser.parse_args()

def save_links(success, failed, outfile):
    with open(args.outfile, 'w') as f:
        for link in success:
            f.write(link + '\n')
        zippy.logger.info('All download links saved at {}'.format(outfile))

    with open("failed.log", 'w') as f:
        for link in failed:
            f.write(link + '\n')
        if len(failed) > 0:
            zippy.logger.info('All failed links saved at failed.log')

if __name__ == "__main__":

    args = load_args()

    if args.filecrypt_url is not None:
        urls = loaders.load_from_filecrypt(args.filecrypt_url)
    elif args.dlcfile is not None:
        urls = loaders.load_from_dlcfile(args.dlcfile)
    elif args.infile is not None:
        urls = loaders.load_from_file(args.infile)
    else:
        urls = loaders.load_from_terminal()

    if len(urls) == 0:
        print('[*] No URLS found!')
        exit(1)

    start = time.time()
    engine = JSEngine
    if args.engine == 'text':
        engine = TextEngine

    zippy = ZippyParser(engine=engine)
    links, fails = zippy.parse_links(urls)
    end = time.time()
    print('Time taken: {:.3f}s'.format(end - start))
    save_links(links, fails, args.outfile)

it's a bit janky, but it works

I've tried your suggestion. New links generated are correct. Just a remember: the links generated are valid only for the IP you're making the request. Example: using my IP I have 'https://www76.zippyshare.com/d/JCgrgF8b/ 114660 /namehere. part23.rar ', then using a different IP I have 'https://www76.zippyshare.com/d/JCgrgF8b/ 18 /namehere. part23.rar ' for the same part of the file. In fact if I try to open the above links in my browser, only the last one generated with the current IP will make the content directly downloadable.

in any case, next month all of this becomes obsolete https://blog.zippyshare.com/?p=1211 ;(

Sad to hear about zippyshare closing down. I fixed the script one last time. I hope you get some utility out of it.


Better customization of char related conversions

I have project with broken headers where char/unsigned char is used interchangeably. Also library does not support UTF8 only plain ASCII.
This PR allows to configure char/unsigned char conversion in whatever way you want.

Thanks for contributing to Chet!
And thanks for following my coding guidelines. Much appreciated!

Wow. My first pull request written in Pascal. And my first PR with code inside.
Thank's for great tool - this projects saved me ton of tedious work.
And actually thanks to Embarcadero for Community Edition. I am still on XE2 so without CE I would not be able to  write this.


show closed and reopen markers inline with comments

just as it is on Github.

Is this still an issue in the current build? If so, can you show some screenshots to demonstrate the issue?

Just like above this commit you see that I have closed and opened the issue.
This is not displayed in the android client.

I'm currently working on this, should have a PR ready by tomorrow

nice :)

 Something like this?

exactly. looks really nice :)

Maybe the icons should be red for close and green for reopen, not sure what the styleguide is here.

This is with color (there is a purple one for merged)

looks much better


Add patch method, fix param quoting (check for list), add access to s…

…ettings and ssl subsetting (more to be added easily if this style is adapted).
Fix for #5.

looks great, but i think that ssl should be an attr of settings (i.e. settings.ssl instead of settings_ssl.

@gnowxilef I guess this pull request is outdated now, will close now. Are you planning to add the subsettings in the style you suggested?

haven't needed it yet, but when i get a chance to yeah.


Robyn version 0.9.0 doesn't work on Mac M1 Models

Error Log :
Finished dev [unoptimized + debuginfo] target(s) in 0.14s
📦 Built wheel for CPython 3.9 to /Users/shivaylamba/Desktop/robyn/target/wheels/robyn-0.9.0-cp39-cp39-macosx_11_0_arm64.whl
💥 maturin failed
  Caused by: pip install in /Users/shivaylamba/Desktop/robyn/venv failed running ["-m", "pip", "--disable-pip-version-check", "install", "--force-reinstall", "/Users/shivaylamba/Desktop/robyn/target/wheels/robyn-0.9.0-cp39-cp39-macosx_11_0_arm64.whl"]: exit status: 1
--- Stdout:

--- Stderr:
ERROR: robyn-0.9.0-cp39-cp39-macosx_11_0_arm64.whl is not a supported wheel on this platform.
---

@messense , The PyPi installation on M1 Mac has stopped working for some reason. By any chance do you happen to know why?

It works on my M1 Mac, you may need to upgrade pip pip install -U pip.
❯ maturin develop
⚠  Warning: `build-backend` in pyproject.toml is not set to `maturin`, packaging tools such as pip will not use maturin to build this project.
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.9 at /Users/messense/.pyenv/versions/3.9.2/envs/test/bin/python
Ignoring uvloop: markers 'platform_machine == "x86_64"' don't match your environment
Ignoring uvloop: markers 'platform_machine == "i686"' don't match your environment
Requirement already satisfied: watchdog==2.1.3 in /Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages (2.1.3)
Requirement already satisfied: multiprocess==<IP_ADDRESS> in /Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages (<IP_ADDRESS>)
Requirement already satisfied: uvloop==0.16.0 in /Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages (0.16.0)
Requirement already satisfied: dill>=0.3.4 in /Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages (from multiprocess==<IP_ADDRESS>) (0.3.4)
warning: unused variable: `url`
  --> src/server.rs:52:9
   |
52 |         url: String,
   |         ^^^ help: if this is intentional, prefix it with an underscore: `_url`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `port`
  --> src/server.rs:53:9
   |
53 |         port: u16,
   |         ^^^^ help: if this is intentional, prefix it with an underscore: `_port`

warning: unused variable: `ctx`
  --> src/web_socket_connection.rs:21:27
   |
21 |     fn started(&mut self, ctx: &mut WebsocketContext<Self>) {
   |                           ^^^ help: if this is intentional, prefix it with an underscore: `_ctx`

warning: unused variable: `handler`
  --> src/web_socket_connection.rs:48:43
   |
48 |                     PyFunction::CoRoutine(handler) => {
   |                                           ^^^^^^^ help: if this is intentional, prefix it with an underscore: `_handler`

warning: unused variable: `text`
  --> src/web_socket_connection.rs:60:34
   |
60 |             Ok(ws::Message::Text(text)) => {
   |                                  ^^^^ help: if this is intentional, prefix it with an underscore: `_text`

warning: unused variable: `router`
  --> src/web_socket_connection.rs:84:21
   |
84 |                 let router = &self.router;
   |                     ^^^^^^ help: if this is intentional, prefix it with an underscore: `_router`

warning: unused variable: `number_of_params`
  --> src/web_socket_connection.rs:86:21
   |
86 |                 let number_of_params = &self.router.get("close").unwrap().1;
   |                     ^^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_number_of_params`

warning: unused variable: `handler`
  --> src/web_socket_connection.rs:97:43
   |
97 |                     PyFunction::CoRoutine(handler) => {
   |                                           ^^^^^^^ help: if this is intentional, prefix it with an underscore: `_handler`

warning: 8 warnings emitted

    Finished dev [unoptimized + debuginfo] target(s) in 0.08s
📦 Built wheel for CPython 3.9 to /Users/messense/Projects/robyn/target/wheels/robyn-0.9.0-cp39-cp39-macosx_11_0_arm64.whl
🛠  Installed robyn-0.9.0
❯ pip --version
pip 21.3.1 from /Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages/pip (python 3.9)

@shivaylamba Can you upload the wheel here? Please also paste the output of pip debug --verbose here.

robyn-0.9.0-cp39-cp39-macosx_11_0_arm64.whl.zip

pipoutput.txt
Output of the pip debug --verbose command @messense

pipoutput.txt Output of the pip debug --verbose command @messense

There are no arm64 tags in this, so I think you're using a x86_64 Python.
❯ cat Downloads/pipoutput.txt | grep arm | wc -l
       0

Here is my pip debug --verbose output
❯ pip debug --verbose | grep arm
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
  cp39-cp39-macosx_12_0_arm64
  cp39-cp39-macosx_11_0_arm64
  cp39-abi3-macosx_12_0_arm64
  cp39-abi3-macosx_11_0_arm64
  cp39-none-macosx_12_0_arm64
  cp39-none-macosx_11_0_arm64
  cp38-abi3-macosx_12_0_arm64
  cp38-abi3-macosx_11_0_arm64
  cp37-abi3-macosx_12_0_arm64
  cp37-abi3-macosx_11_0_arm64
  cp36-abi3-macosx_12_0_arm64
  cp36-abi3-macosx_11_0_arm64
  cp35-abi3-macosx_12_0_arm64
  cp35-abi3-macosx_11_0_arm64
  cp34-abi3-macosx_12_0_arm64
  cp34-abi3-macosx_11_0_arm64
  cp33-abi3-macosx_12_0_arm64
  cp33-abi3-macosx_11_0_arm64
  cp32-abi3-macosx_12_0_arm64
  cp32-abi3-macosx_11_0_arm64
  py39-none-macosx_12_0_arm64
  py39-none-macosx_11_0_arm64
  py3-none-macosx_12_0_arm64
  py3-none-macosx_11_0_arm64
  py38-none-macosx_12_0_arm64
  py38-none-macosx_11_0_arm64
  py37-none-macosx_12_0_arm64
  py37-none-macosx_11_0_arm64
  py36-none-macosx_12_0_arm64
  py36-none-macosx_11_0_arm64
  py35-none-macosx_12_0_arm64
  py35-none-macosx_11_0_arm64
  py34-none-macosx_12_0_arm64
  py34-none-macosx_11_0_arm64
  py33-none-macosx_12_0_arm64
  py33-none-macosx_11_0_arm64
  py32-none-macosx_12_0_arm64
  py32-none-macosx_11_0_arm64
  py31-none-macosx_12_0_arm64
  py31-none-macosx_11_0_arm64
  py30-none-macosx_12_0_arm64
  py30-none-macosx_11_0_arm64

You can run this to confirm your Python's architecture:
❯ python -c 'import sysconfig; print(sysconfig.get_platform())'
macosx-11.2-arm64

@messense

I uninstalled Python and reinstalled it. It seems to work now. Thanks for your help @messense

Thanks for all the help @messense 😃✨


Empty environment variables are not maintained

Description
I have a service with just one replica. This service contains multiple environment variables, some of them are empty.
If I scale the service to 0 and then to 1 again, the empty environment variables disappear from the list and on top of that, they are not being propagated to the new replica.
Steps to reproduce the issue:

Deploy a service with 1 replica and empty env. variables.
Scale the service to 0.
Scale the service to 1.
You can observe that empty env. variables are not maintained.

Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead?
I was expecting that all environment variables, even the empty ones, are maintained and propagated.
Technical details:

Portainer version: v1.12.4
Portainer Docker image tag (latest/arm/windows...): latest
Target Docker version (the host/cluster you manage): 17.04.0-ce
Target Swarm version (if applicable): 1.28
Platform (windows/linux): linux
Browser: safari (osx)

When you're talking about empty env. variables, are you talking about env. entries with no name, no value or both?
At the moment, Portainer will not take into account any variable with either no name or no value. So, the problem is not that empty env. vars are not maintained after a scale/service update action.
E.g. you should be able to reproduce it this way:

Deploy a service with empty env. variables
Inspect the created service, empty env. vars are not maintained

What would be the use case to support empty env. vars?

Hi @deviantony,
I was talking about env. entries without a value.
In my case, I have an env variable without a value in some containers and with value in some others (same stack).
This variable is required regardless the value for starting supervisor (http://supervisord.org/configuration.html#environment-variables). It's used inside the configuration and if it's not defined, supervisor cannot start. If it's defined, but without a value, it is ok.
Hope it helps.

Ok, we can support env entries without a value.


Search Marketplace for Extensions for 'language Id' is broken

The recommendation filtering is broken and currently all language extensions are shown ignoring the actual language id.

I've removed the action for now from the menu.

Looks like this commit broke this behaviour. There was a command that shows extensions with given language id. But this change added a new action with same id as the other, to show extensions with language category.


Can you add rtp protocol

When I use RTSP and RTMP with high latency, my Android camera has poor streaming latency. It is best to use the RTP protocol to reduce latency, so can you add the RTP protocol? Thanks

Hello,
Do you have a way to test it?
I need an environment to test with RTP protocol

Hello,
Do you have a way to test it? I need an environment to test with RTP protocol

Of course, you can use this app to display your RTP video stream for development testing
test app download：
https://drive.google.com/file/d/10LN0UyKvMGZju3FasQVBoWaQmL6UILx3/view?usp=sharing

And  also can view the source code about this app:  https://github.com/Consti10/LiveVideo10ms

I'm not able to connect that app to ffmpeg using rtp. Also, I can't find a server that allow RTP stream raw without RTSP. I think that implement this protocol if I don't know the usage of it is not a good idea.
This will be stopped until a clear way to test it is explained


Unable to reproduce model results

Hello! Thank you for releasing the training script as requested in #5.
Unfortunately, I haven't had any luck reproducing the results from the paper using the provided training script and config. I slightly modified the config in language_table/train/configs/language_table_sim_local.py so that checkpoints save every 50,000 steps and the training stats print every 5,000 steps, otherwise it's all the same.
The loss decreases over the first 1,000 steps and then fluctuates around loss=0.9 and loss_std=0.5 for the remaining ~700,000 steps that I trained for. As expected, the model doesn't perform well using the evaluation script.
Something strange that I noticed is that the LR stayed the same throughout the training, which is unexpected since the training uses the adam optimizer.
I noticed that the batch size was commented out for TPUs, was this config built only to be used with TPUs?
Command: nohup python language_table/train/main.py --config=./language_table/train/configs/language_table_sim_local.py --workdir={PATH}/data/language_table_train/ &
Start of training:

Start of output log:

I am using an NVIDIA GeForce RTX 3090, and I'm currently training again with a larger batch size.
Please let me know if you would like any additional info or the full training log. Thank you!

Edit: The loss seems to get stuck and the LR doesn't change when the batch size is 128 as well.

Hey Jacob, I have encountered exactly the same problem as you. My loss also quickly dropped to around 0.9 and hovered. My global batchsize is set to 64, and lowering the learning rate does not change this situation. Can you provide us with some help? Thank you very much! @ayzaan

Hello,
The actual models we train use a much larger batch size of 4096. The config file language_table_sim_local.py is mostly intended as a reference for testing that the training runs locally.
A config that matches performance from the paper has the following:
config.data_normalization_num_samples = 3200
# 128 chips; global batch size = 4096; 32 per device.
config.per_device_batch_size = 32
config.replay_capacity = 50_000

The per_device_batch_size is 32 with 128 chips total (8x8 TPU), giving global batch size of 4096. In the experiments we ran for the paper (Table VI), cutting the batch size in half dropped performance by about 5%, so wouldn't be surprised if dropping it to 128 had a much larger effect. @jacob-zietek Do you have an idea of the largest batch size that you could fit on a 3090? If you'd like I can try to launch a training run with that batch size as a reference point for you to reproduce. Best would be to use 4096 but that will probably require more accelerator resources.
Also, wouldn't hurt to increase both the data_normalization_num_samples and replay_capacity, those values above are what we used in the paper. I'll check in a template config with these values for reference.
Let me know if you have any more questions.

Hi @ayzaan, thanks for the quick response!
Thank you for clearing that up. Unfortunately, the largest batch size that can fit on a single 3090 is 128. it peaks at ~22.5GB of VRAM during training. I would appreciate it if you could try to train with 128!
I see the replay_capacity is tied to the shuffle buffer size here. Shouldn't this be set to (Devices * Batch size) instead of 50_000? Perhaps I'm misunderstanding the purpose of the buffer.
I will try to train with a larger data_normalization_num_samples and replay_capacity now, thank you again!

replay_capacity is perhaps not the best name, it does just mean the shuffle buffer size in the data pipeline - to ensure the input data is shuffled adequately. We also trained in a distributed setting and had distributed data workers, which each shuffled with 50k elements and fed into training. In a smaller setup like yours I'd probably try to crank it as high as possible while still fitting on your machine RAM.
I'll launch a training run with a small batch size like 128 and see what happens. One hunch is that you should also decrease the learning rate significantly.

I see, thanks for clearing that up! I'll experiment with increasing the capacity and decreasing the LR if this training run fails.

replay_capacity is perhaps not the best name, it does just mean the shuffle buffer size in the data pipeline - to ensure the input data is shuffled adequately. We also trained in a distributed setting and had distributed data workers, which each shuffled with 50k elements and fed into training. In a smaller setup like yours I'd probably try to crank it as high as possible while still fitting on your machine RAM.
I'll launch a training run with a small batch size like 128 and see what happens. One hunch is that you should also decrease the learning rate significantly.
Thank you for detailed explanation. Can you share the approximate convergence value of loss when you train on language_table and language_table_sim. Thanks!

Thank you for detailed explanation. Can you share the approximate convergence value of loss when you train on language_table and language_table_sim. Thanks! @ayzaan

Here are a couple of results on language_table_sim. Overall it seems performance does decrease significantly with smaller batches, although you should still see some successful rollouts during eval.
black is the baseline with batch_size=4096
orange is batch_size=256 with LR=1e-4
green is batch_size=128 with LR=5e-5.

And success rates:

Thanks a lot for running additional training runs! I will use batch_size=128 with LR=5e-5


Rework the drupal.scss .element-invisible

@weerd Perusing through old issues (nostalgia!), is this looking better now?

¡Es bueno! :+1:


fix: remove accel_brake_map_calibrator

Description
Moved accel_brake_map_calibrator back to awf/autoware.universe

Related links
https://github.com/autowarefoundation/autoware.universe/pull/6453

Tests performed

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

In the linked PR, it says "Revert the accel_brake_map_calibrator from awf/autoware_tools since lexus_description depends on this package", but why is it necessary in lexus_description? It shouldn't be launched in normal driving, right?


Pluralsight password having "" character.

Checklist

[Yes] I'm reporting a broken site support issue
[ Yes] I've verified that I'm running youtube-dl version 2019.04.30
[Yes] I've checked that all provided URLs are alive and playable in a browser
[Yes] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[Yes] I've searched the bugtracker for similar bug reports including closed ones
[Yes] I've read bugs section in FAQ

Verbose log
-->
youtube-dl --username "REDACTED" --password "Alphabets replaced with * only symbols kept as is }~**+8**\**" --verbose --sleep-interval 120 "https://app.pluralsight.com/library/courses/operationalizing-cyber-threat-intel-pivoting-hunting/table-of-contents"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--username', u'PRIVATE', u'--password', u'PRIVATE', u'--verbose', u'--sleep-interval', u'120', u'https://app.pluralsight.com/library/courses/operationalizing-cyber-threat-intel-pivoting-hunting/table-of-contents']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.30
[debug] Python version 2.7.16 (CPython) - Linux-5.0.0-13-generic-x86_64-with-Ubuntu-19.04-disco
[debug] exe versions: none
[debug] Proxy map: {}
[pluralsight:course] operationalizing-cyber-threat-intel-pivoting-hunting: Downloading JSON metadata
[download] Downloading playlist: Operationalizing Cyber Threat Intel: Pivoting & Hunting
[pluralsight:course] playlist Operationalizing Cyber Threat Intel: Pivoting & Hunting: Collected 36 video ids (downloading 36 of them)
[download] Downloading video 1 of 36
[pluralsight] Downloading login page
[pluralsight] Logging in
ERROR: Unable to log in; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 528, in extract
    self.initialize()
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 432, in initialize
    self._real_initialize()
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pluralsight.py", line 165, in _real_initialize
    self._login()
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pluralsight.py", line 215, in _login
    raise ExtractorError('Unable to log in')
ExtractorError: Unable to log in; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

I suspect " \ " character in the password to be causing this issue. I have  used the same command line with the same switches to successfully connect and login using different name and password.
My password without numbers and alphabets has the following:      }~+\

Since you're on NIX, try replacing the double quotes surrounding your pw with single quotes.

I am using the same command line and switches (including double quotes) with different password and it works perfect. Once this is one, I will try with single quotes and the PWD i had.
Hopefully, I will find my first bug :D

youtube-dl --username<EMAIL_ADDRESS>--password '}~+8\' --verbose --sleep-interval 120 "https://app.pluralsight.com/library/courses/operationalizing-cyber-threat-intel-pivoting-hunting/table-of-contents" [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'--username', u'PRIVATE', u'--password', u'PRIVATE', u'--verbose', u'--sleep-interval', u'120', u'https://app.pluralsight.com/library/courses/operationalizing-cyber-threat-intel-pivoting-hunting/table-of-contents'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2019.04.30 [debug] Python version 2.7.16 (CPython) - Linux-5.0.0-13-generic-x86_64-with-Ubuntu-19.04-disco [debug] exe versions: none [debug] Proxy map: {} [pluralsight:course] operationalizing-cyber-threat-intel-pivoting-hunting: Downloading JSON metadata [download] Downloading playlist: Operationalizing Cyber Threat Intel: Pivoting & Hunting [pluralsight:course] playlist Operationalizing Cyber Threat Intel: Pivoting & Hunting: Collected 36 video ids (downloading 36 of them) [download] Downloading video 1 of 36 [pluralsight] Downloading login page [pluralsight] Logging in ERROR: Unable to log in; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last):   File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info     ie_result = ie.extract(url)   File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 528, in extract     self.initialize()   File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 432, in initialize     self._real_initialize()   File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pluralsight.py", line 165, in _real_initialize     self._login()   File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pluralsight.py", line 215, in _login     raise ExtractorError('Unable to log in') ExtractorError: Unable to log in; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Still the same thing. (YAY and sorry).

For any work on this you must provide account credentials.

I can provide the credentials, could you confirm who should I direct message those to?


Batch job start

When adding a batch job, why do you want to wait until after the completion of the task to grab? For example, I added a URL 5W, I read the log is to wait until the completion of the 5W to start?

I think asynchronous add tasks, asynchronous scheduling better, I do not know your intention?

For simplification, there is only one process running.

Thanks to answer, so, can only wait for the completion of all tasks can be executed to grab, a bit unreasonable, right?

your "wait until the completion of the 5W to start" means the starting up stage that scheduler select tasks from database?
as long as scheduler only has one process, it's not possible to switch to dispatching stage before finish it's job, it will lost its cursor of database.
Correct me if I misunderstand you.

I want to grab URL: 5W
for url in urls:
self.crawl(url)
But this URL 5W did not add, it will not be scheduled.
But asynchronous add tasks, asynchronous scheduling, more reasonable?

You mean you added 50k urls within 1 response/task/callback?
well, for this case, it's true that all 50k urls is put in one "new_request" pack, yes, it can be separated.

Sorry, I think, and so on all the tasks to add a long time, I hope to add tasks at the same time can be carried out, I do not know how you put the plan to do?

separate them into 1k package.
BTW: adding 50k urls is not a high frequency job, comparing to the fetching time, start crawling same time as added will not make much difference.

I now know that millions of URL, want to grab a quick, follow up may also be updated regularly, perhaps a Monday times, perhaps in January. So URL is already clear.

Fix: a week or a month

Yes, I can add the feature make it start crawling quicker. but note that, it's not adding weekly/monthly job using cronjob in pyspider. You cannot control which day/hour/second in the month it should trigger your task.

This issue should be fixed in https://github.com/binux/pyspider/commit/e59bd63b56270d6a4b85905327ae9d726cc4cc75#diff-0d49db77523b2eaebf35f1f83bff4e0cL178

How fast can you add up to a million tasks?
    def on_start(self):
        for url in all_urls:  # million url
            self.crawl(url, callback=self.parse_page, retries=10)

depends on what database you are using: https://github.com/binux/pyspider/issues/138#issuecomment-77146026

mongodb

How to split the last one million URL packets, and quickly save to mongodb?

I think that would take much more time to fetch rather than save to database.

In fact, the purpose is very simple, there are millions of URL, because the storage for a long time, the storage will be completed before scheduling, feeling too slow. How to fast storage and scheduling, and half a month or January update? There are no similar examples?

you can create some "virtual" task, and submit 10k each time. and set detail page priority higher than the virtual task. pyspider will automatically load next 10k links after detail page job finished.
def on_start(self):
    for i in range(100):
        self.crawl('data:;fake_url_%s' % i, save=i, callback=self.load_urls)

@config(priority=0)
def load_urls(self, response):
    for url in self.database.get(offset=response.save * 10000, limit=10000):
        self.crawl(url, callback=self.detail_page)

@config(priority=1)
def detail_page(self):
    pass

Thank you for your plan, there is a question, according to your method, but the first time to add the URL 4W. Why not add 1W to grab the next 1W?

where the 4w can from? cannot understand.

In your way, the first task was to add the URL 4W, then begin to crawl.
4W crawl completed, But it loads 20W URL.
Why not load 1W, grab 1W? continue loading 1W?

note @config(priority=0), priority made it sure that detail pages crawled before load_urls, but as tasks is running concurrently, it maybe more than 1 load_urls is running at the first time.


Nested Tag Unions with abilities crash the compiler

Just put this in a file and try to compile it:
OtherType k : [ OtherType k ] | k has Hash.Hash
SomeType k : [ SomeType (OtherType k) ] | k has Hash.Hash

someVal : SomeType (OtherType I32)
someVal = SomeType (OtherType 32)

The result is:
thread '<unnamed>' panicked at 'Able var in type arg for [TagName('OtherType') <1624>] - this is a bug in the solver, or our understanding', crates/compiler/solve/src/solve.rs:2612:83

Notes:

This only fails when the type is specified
It also only fails with the nesting, someVal : OtherType I32 works fine

This is fixed by https://github.com/roc-lang/roc/pull/4305.


Move CI to CircleCI

In an effort of consolidation, I'm trying to get it to work from CircleCI.
Every push is checked for syntax and linting.
Pushes to master with a tag that matches metadata.json will automatically push the module to the Puppet Forge.

https://github.com/SineQuaNonSoftware/puppet-proxmox/pull/16

Merged.

Maybe I shouldn't have. Deploy is borked.

As far as I can gather, that's the check you talked about regarding tag vs metadata for build ? Shouldn't it just "not deploy" in that case instead of the step breaking ?

Apparently it's just "the way the orb does it" and not a choice on our part.
That means we will end up ignoring every test result on master when we're not pushing a release tag.

Yeah, I think I can specify the step is allowed to fail (yellow ball instead of the green tick).
I didn't read the publish code in detail. I just wanted to have the basic syntax checks for every branch and PR.
I'll need to figure out how that exactly works to make it try and deploy only if the tag is there.
Creating a new issue to fix the deployment part.


chore: add vite v5 to peer dependencies

resolves #55

While updating the peer dependencies will fix the npm installation in consumers, don't we want to also bump the version of vite to v5 that the source is compiling and using? So the plugin is tested against the version its declaring its compatible with

@TomPlum you're totally right. I've updated the PR

@TomPlum can you please review? I need this PR to get my stuff working

Hello @daiky00, please note that @TomPlum isn't responsible for managing this repository, so he can't merge your pull request. The only person with merge authority here is @davidmyersdev. However, repeatedly contacting him probably won't expedite the process. I'm sure he's already aware of the issue and he will look into it, when he has the time.
If you need a quick solution, consider forking the package or implementing the temporary fix suggested in issue #55. Persistently nudging others is unlikely to accelerate the resolution.

Hey, @maxnowack! Thanks so much for the PR. I want to make sure we're still testing other versions of Vite, so I'll likely need to modify this a bit after I merge it. That said, I'll try to get an initial release out today to fix the peer dependency and worry about supporting multiple versions after that.

@davidmyersdev thanks for merging. I already have an idea in mind how testing multiple versions of peer dependencies can be achieved. Just let me now if you want me to open a pull request for that 🙂

No problem at all! 😄 This was released in v0.17.0.


Null Reference passing object to LatLngBounds.contains

How to reproduce

Leaflet version I'm using: 1.0.1 & 1.0.2
Browser (with version) I'm using: Firefox 49.02 and Edge 25
OS/Platform (with version) I'm using: MacOS 10.11.6 for Firefox/Windows 10 for Edge
call LatLngBounds.contains with an object-style LatLng, e.g. {lat:0,lon:0}

What behaviour I'm expecting and which behaviour I'm seeing
expecting: the function to return true or false
seeing: a null reference exception
Minimal example reproducing the issue
http://codepen.io/plaidphantom/pen/YpVZap
I noticed this issue originally while using 1.0.1, but putting together the CodePen I tested it with 1.0.2 and get identical behavior.
According to the LatLng documentation i think this should work, as there's nothing "noted otherwise" on .contains

#5135 should do the trick.


failing GFI requests with special characters (umlauts) when lxml is installed

We have started to use XLST scripts to transform GetFeatureInfo responses of a source WMS. Generally, this works fine and as expected. However, some of the XML responses contain attributes with special characters (in our case: German umlauts) which lead to misformed XML responses. This behavior also shows when the lxml package is installed but no actual XSLT scripts are enabled.
It seems that the proxying of GFI responses differs depending on the availability of the lxml package.
Context
The source WMS GetFeatureInfo XML is:
<?xml version="1.0" encoding="UTF-8"?>
<FeatureInfoResponse xmlns:esri_wms="http://www.esri.com/wms" xmlns="http://www.esri.com/wms">
    <FIELDS OBJECTID="21" AttributeWithÄ="ö" />
</FeatureInfoResponse>

✅ MapProxy's proxied XML without lxml installed is:
<?xml version="1.0" encoding="UTF-8"?>

<FeatureInfoResponse xmlns:esri_wms="http://www.esri.com/wms" xmlns="http://www.esri.com/wms">
    <FIELDS OBJECTID="21" AttributeWithÄ="ö" />
</FeatureInfoResponse>

❌ MapProxy's proxied XML with lxml installed is:
<FeatureInfoResponse xmlns:esri_wms="http://www.esri.com/wms" xmlns="http://www.esri.com/wms">
    <FIELDS OBJECTID="21" AttributeWithä="ö" />
</FeatureInfoResponse>

Note the broken attribute at ä - also, the XML version tag is omitted.

Expected Behavior
I would expect both proxied GetFeatureInfo responses to be identical when no XSLT transformations are configured.
Actual Behavior
When the lxml package is installed, special characters are being transformed into their unicode codes which results in malformed XML documents (when the special characters occur, e.g. as attribute name as in the example above)
Your Environment
I use the stable kartoza/docker-mapproxy base image with MapProxy 1.14.0.
lxml is installed in the Dockerfile via pip install lxml.

could you provide a url / layer to test?

Unfortunately, our WMS services are not public. I was able to reproduce the issue with a public WMS, the NRW Unfallatlas:
The GFI responses of the layer Unfallorte have an attribute Fußgänger with different special characters (ßand ä, in this case).
Here's the mapproxy.yaml configuration for that layer:
caches: {}
layers:
- layers:
  - name: Unfallorte
    sources: [Unfallorte_wms]
    title: Unfallorte
  srs: ['EPSG:4326', 'EPSG:25832']
  title: Layers
services:
  demo:
  wms:
    md:
      title: Unfallatlas
    on_source_errors: raise
sources:
  Unfallorte_wms:
    coverage:
      bbox: [5.567798, 47.187915, 15.55503, 55.038438]
      srs: EPSG:4326
    req:
      layers: Unfallorte
      transparent: true
      url: https://www.wms.nrw.de/wms/unfallatlas?
    supported_srs: ['EPSG:4326']
    type: wms
    wms_opts:
      featureinfo: true
      legendurl: https://www.wms.nrw.de/wms/unfallatlas?request=GetLegendGraphic%26version=1.3.0%26format=image/png%26layer=Unfallorte
      version: 1.3.0

This GFI request (format text/xml) on the source WMS works:
https://www.wms.nrw.de/wms/unfallatlas?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetFeatureInfo&BBOX=53.73525498559196478,10.35376169783363487,53.74283750983437358,10.35999436611201041&CRS=EPSG:4326&WIDTH=748&HEIGHT=910&LAYERS=Unfallorte&STYLES=&FORMAT=image/png&QUERY_LAYERS=Unfallorte&INFO_FORMAT=text/xml&I=306&J=493
While the same request on a local mapproxy instance (here on localhost:8081) fails returning a malformed xml:
http://localhost:8081/service?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetFeatureInfo&BBOX=53.73525498559196478,10.35376169783363487,53.74283750983437358,10.35999436611201041&CRS=EPSG:4326&WIDTH=748&HEIGHT=910&LAYERS=Unfallorte&STYLES=&FORMAT=image/png&QUERY_LAYERS=Unfallorte&INFO_FORMAT=text/xml&I=306&J=493
This is the returned malformed xml:
<FeatureInfoResponse xmlns="http://www.esri.com/wms" xmlns:esri_wms="http://www.esri.com/wms">
	<FIELDS OBJECTID="508698" Shape="Point" UJAHR="2018" Rad="0" PKW="1" Fußgänger="0" Kraftrad="0" Sonstige="0" GKFZ="1" Unfallkategorie="Unfall mit Leichtverletzten"/>
</FeatureInfoResponse>

Note: The lxml package has to be installed for this issue to occur.

Thanks for the details, i will have a look at it!

Found a fix in the mentioned PR


Keep getting "Error in plugin: read |0: file already closed"

Hi,
I am testing the plugin in container. When I run it manually it works like a charm.
telegraf@ba42cee71747:~$ /usr/local/bin/python /usr/local/bin/telegrafbacnet --config /etc/telegrafbacnet/config.toml
bacnet,deviceAddress=<IP_ADDRESS>,objectType=analogInput,objectInstanceNumber=36,deviceName=E09_25 presentValue=24.059999465942383<PHONE_NUMBER>929842419
bacnet,deviceAddress=<IP_ADDRESS>,objectType=analogInput,objectInstanceNumber=36,deviceName=E09_25 presentValue=24.059999465942383<PHONE_NUMBER>928126242

However, when I run it from telegraf with the example configuration, it receives TERM signal and nothing is in output. In log there is an error:
2023-09-12T05:38:34Z I! [inputs.execd] Starting process: telegrafbacnet [--config /etc/telegrafbacnet/config.toml]
2023-09-12T05:38:34Z D! [agent] Stopping service inputs
2023-09-12T05:38:39Z E! [inputs.execd] stderr: "===== TERM Signal, 12-Sep-2023 05:38:39"
2023-09-12T05:38:44Z E! [inputs.execd] Error in plugin: error reading stderr: read |0: file already closed
2023-09-12T05:38:44Z E! [inputs.execd] Error in plugin: read |0: file already closed
2023-09-12T05:38:44Z I! [inputs.execd] Process /usr/local/bin/telegrafbacnet shut down
2023-09-12T05:38:44Z D! [agent] Input channel closed
2023-09-12T05:38:44Z D! [agent] Stopped Successfully
2023-09-12T05:38:44Z E! [telegraf] Error running agent: input plugins recorded 3 errors

I tried to change collection interval and restart_delay but without success.
Can you point me in the right direction, what's wrong?
Thanks Jan

It was probably only my misunderstanding of the configuration.


[fix] increase hammer icon size

Before
After

@mittalyashu Can you explain why this refactor was needed? @Andreto created this icon in series in #118 with some other tool icons, so each tool have a consistent size and handle.

@mittalyashu The icon was created to be consistent with other icons with handles and especially axe and gavel. The handle could maybe be a bit longer but otherwise, I don't think that the sizing is a problem.

I close this one for now, if you have a clear explanation we can consider opening this PR again.
You still open to comment here.


Chopstick crash on latest Moonbeam runtime (RT2500)

(RT2500 is only on alphanet, not yet on Moonriver/Moonbeam)
Command:
node_modules/.bin/ts-node packages/chopsticks/src/cli -c configs/moonbase-alpha.yml
This is forking Moonbase Alphanet (RT2500).
 I tried a few balances.transferKeepAlive and it is running fine.

Thank you @xlc I just retried and it works now. I suspect it might have been an issue with polkadotjs api


select only resolved comments

@yegor256 this pr fixes https://github.com/zerocracy/judges-action/issues/308
I think it is more logical to fix the Fbe.github_graph.resolved_conversations method, because this function should only return resolved conversations, but now it does not. Thus, we will not have to fix anything in judges-action, other than upgrading the fbe version.

@Suban05 thanks!

@rultor release, tag is 0.0.59

@rultor release, tag is 0.0.59

@yegor256 OK, I will release it now. Please check the progress here

@rultor release, tag is 0.0.59

@yegor256 Done! FYI, the full log is here (took me 24min)

@Suban05 Hey there! 👋 Thanks for your contribution, but it looks like we've got some room for improvement. You earned +4 points this time: +16 base, but we had to deduct points for missing code review and comments. Your 12 hits-of-code were a bit low too (-8 points). We bumped you up to the minimum 4 points to keep you going. Remember, more hits-of-code (aim for 20+) and getting your code reviewed with comments can really boost your score! Keep the contributions coming – you're doing great overall with a balance of +532!


Add function to reset lwm2m-context of a client

Signed-off-by: Pehr Nord<EMAIL_ADDRESS>#663

Does lwm2m_context.nextMID need to be reset as well?

Does lwm2m_context.nextMID need to be reset as well?

Maybe, you check RFC7252, Section 4 about the usage of the MID. Then you may describe the issue you consider in your case and how a reset or not reset will solve that.
In my experience, don't reset it. I only know one issue with MIDs after client restart: you may hit an coap-exchange, which is still kept for deduplication. But the probability of that doesn't change, if you reset the MID in advance. You may just check, if the response (if piggybacked) matches by the MID but not by the token. If that happens after a device restart, you may consider to reset the device MID by a new random value. Or wait for 247s (EXCHANGE_LIFETIME) in order to ensure, that the server's deduplication store is cleaned up.

@pehrnord Will you rework this PR?


[master]Set cache for volumes list of the host

Signed-off-by: Xiao Feng Ren<EMAIL_ADDRESS>

@bjhuangr @haoqf  please help review.

@bjhuangr @SeanHQF please help review. have verified locally.

@jichenjc @SeanHQF  please help review. have tested locally

@trenxiaof pls solve the conflict, thanks

@trenxiaof CI failed.

@jichenjc can you help review? thanks

@bjhuangr can you help check this?

@trenxiaof any plan to update this patch?

@trenxiaof any plan to update this patch?

Yes, will update it with your comments

@huangrui add self.disk_pool to save the old disk_pool, if the disk_pool in the config file or called are  changed, the volumes list will be updated for this new disk_pool.

@bjhuangr please help review


OptionRow: Fix label overlapping with button for long texts

Fixes #2779

Can you provide a screen-shot?
I am not sure if doing a flex: 1 will properly resize it.
I was thinking our two options are to either ellipsize the text ... or to make it multi-line.

Doing flex: 1 does cause a line break:

Cool. I think this is better than an ellipsis.


custom css

custom css
It would be very useful to be able to customize the theme via css.

Hey @DoctorSubtilis, release 0.17.2 included this pull request from the original marktext repository, which adds some custom CSS support to marktext.
Are you having difficulty finding/using this feature, or does it not fulfill your needs?


proposal: print AST value visually

Util now, when we get an AST Value, it's hard to know the underlying details even if debugged because the deep inclusion chain.
I propose to add a function to print the AST value visually. For example:
			Object
	 ______________/  |   \_____________
	/                 |                 \
    "Key":1	          "Key2": "Hello"   "Key3": Object
						      |
						      "Key4": "hello"

The hardness is the align issue which requires a lot of work but very trivial achievement.

I'm likely to put it aside for a long time because it's too trivial and has no apparent benefit...


[Bug]: Index treated as undefined even though it is defined in custom widget code.

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When I attach an index to an event in a custom widget code, in the Events Property in the Edit mode of application, the index is treated as undefined
Steps To Reproduce

On writing a custom code [code for button widget] for a Custom widget, define an index for trigger event.
Click on the button and note event is triggered
However, in the onButtonClick event, index is treated as undefined

Public Sample App
No response
Environment
Release
Issue video log
No response
Version
Cloud

Bumping this issue up to prod because it should have made it's way there by now if it was not fixed


Segmentation fault

Hi
I was trying to run the demo, the demo terminated with the following output.
episode ended with reward -24845.0
Segmentation fault
I am wondering how to solve the "Segmentation fault" part? I am using wsl2 with Ubuntu 20.04, is that related?
Thank you.

Hi,
Can you please post here the exact code you are running (Wildfire?) and the stack trace if possible?
We have not tested with wsl so it might be related, we have tested with native linux, windows and Apple silicon without errors.

Hi,
This is also an issue I have observed recently with wsl as well. I believe the error can be traced to the visualizer internals, e.g. matplotlib or pillow, so it is very likely the error is on their end. We will do more tests and let you know if we come up with a solution. In the meantime, a simple solution may be to run the GymExample without the visualization, which should (hopefully) not raise this error.

Hi,
Yes, running it without rendering gets rid of the error. Thank you for the help.

Thanks for your report. We will keep this issue open for now until we can find a better solution for wsl.

I did have the stack trace available, just posting it here for reference.
(gdb) run /home/leo/ipc2023/demo.py
Starting program: /home/leo/miniconda3/envs/pyrddlgym/bin/python /home/leo/ipc2023/demo.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

[Switching to Thread 0x7fffd1ae0700 (LWP 1619)]
0x00007fffc5ac9efa in ?? () from /usr/lib/wsl/drivers/nvamig.inf_amd64_d36b3e14914fc88f/libnvwgf2umx.so
(gdb) backtrace
#0  0x00007fffc5ac9efa in ?? () from /usr/lib/wsl/drivers/nvamig.inf_amd64_d36b3e14914fc88f/libnvwgf2umx.so
#1  0x00007fffc5ac8c5e in ?? () from /usr/lib/wsl/drivers/nvamig.inf_amd64_d36b3e14914fc88f/libnvwgf2umx.so
#2  0x00007fffc5ac8bd6 in ?? () from /usr/lib/wsl/drivers/nvamig.inf_amd64_d36b3e14914fc88f/libnvwgf2umx.so
#3  0x00007ffff7fa3609 in start_thread (arg=) at pthread_create.c:477
#4  0x00007ffff7d6e133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
The demo code I ran is Wildfire, also posting it here.
from pyRDDLGym import RDDLEnv
from pyRDDLGym import ExampleManager
from pyRDDLGym.Policies.Agents import RandomAgent
ENV = 'Wildfire'
get the environment infos
EnvInfo = ExampleManager.GetEnvInfo(ENV)
set up the environment class, choose instance 0 because every example has at least one example instance
myEnv = RDDLEnv.RDDLEnv(domain=EnvInfo.get_domain(), instance=EnvInfo.get_instance(0))
set up the environment visualizer
myEnv.set_visualizer(EnvInfo.get_visualizer())
set up an example aget
agent = RandomAgent(action_space=myEnv.action_space, num_actions=myEnv.numConcurrentActions)
total_reward = 0
state = myEnv.reset()
for step in range(myEnv.horizon):
# myEnv.render()
action = agent.sample_action()
next_state, reward, done, info = myEnv.step(action)
total_reward += reward
state = next_state
if done:
break
print("episode ended with reward {}".format(total_reward))
myEnv.close()

Thanks for the stack trace. Are you by any chance using a dedicated gpu (e.g. nvidia) and/or any graphical extension for wsl?
This is almost surely a graphical/driver error with matplotlib/pil and wsl/wsl2.
We will look into this, and get back to you if we find a solution.

I do have nvidia gpu installed but I am not sure about wsl. I tried to print out the driver info and the results are in the following
lspci
3ed2:00:00.0 3D controller: Microsoft Corporation Basic Render Driver
64a1:00:00.0 SCSI storage controller: Red Hat, Inc. Virtio filesystem (rev 01)
686a:00:00.0 SCSI storage controller: Red Hat, Inc. Virtio filesystem (rev 01)
9f09:00:00.0 SCSI storage controller: Red Hat, Inc. Virtio filesystem (rev 01)
a96b:00:00.0 System peripheral: Red Hat, Inc. Virtio file system (rev 01)
cae7:00:00.0 SCSI storage controller: Red Hat, Inc. Virtio console (rev 01)
e7c8:00:00.0 3D controller: Microsoft Corporation Basic Render Driver

Hi,
Thanks for providing this info.
We've investigated this and were able to reproduce the problem in WSL2 using hyper-v virtual machine.
It is likely related to a well-known problem between pygame and wsl2. You can try the solutions posted there:
PyGame issue 3260
You can try updating drivers as suggested there, or (ideally) use a different vm (e.g. vmware) if you need visualization on linux.
Since we cannot provide a definitive solution as of yet, we can leave this problem open for now.

Thank you!


Logging to syslog hardcoded for zblacklist/ziterate

The zblacklist and ziterate programs (in 2.1.1 & master) are hardcoded to log to syslog. This is problematic when doing large filtering operations due to creation of a huge amount of log data and CPU usage of the system logging daemons.
zblacklist.c
if (log_init(logfile, conf.verbosity, 1, "zblacklist")) {

ziterate.c
if (log_init(logfile, conf.verbosity, 1, "ziterate")) {

Can this behavior be changed to default to no syslog logging for these tools, or configured via a command-line option or environment variable instead?

I don't see any reason why we can't add --disable-syslog to match ZMap.


Bulk Document Indexing

Description
Location: admin/plugins/meilisearch
As a User, when using the Meilisearch Strapi plugin, and utilizing the Update Button with a 1000+ data-set.
Expected behavior
Document count out of total will increment/transfer.
Currently sitting at: 0/1195 using the Update Feature.
Current behavior
Nothing happens.
When manually un-publishing/republishing an item in the collection, the documents will sync via hook.
Screenshots or Logs

After Manual Publishing:

Environment (please complete the following information):

OS: OSX
MeiliSearch version: v0.23.1
strapi-plugin-meilisearch version: "strapi-plugin-meilisearch": "^0.3.3",
Browser: Chrome etc etc (Brave/Safari)
Strapi version: 3.6.8
Flags: --http-payload-size-limit=1048576000
Resources: 16/32GB of RAM

Other:

Able to upload the Sample Dataset without issue of Movies. (19k+)

{
    "numberOfDocuments": 19546,
    "isIndexing": false,
    "fieldDistribution": {
        "genres": 19456,
        "id": 19546,
        "overview": 19546,
        "poster": 19546,
        "release_date": 19546,
        "title": 19546
    }
}

Area of Interest: strapi-plugin-meilisearch/admin/src/components/Collections.js -> updateCollections.js
// Re-indexes all rows from a given collection to MeilISearch
  const updateCollections = async ({ collection }) => {
    setCollectionsList(prev =>
      prev.map(col => {
        if (col.name === collection)
          return { ...col, indexed: 'Start update...', _isChecked: true }
        return col
      })
    )
    const response = await request(`/${pluginId}/collections/${collection}/`, {
      method: 'PUT',
    })
    if (response.error) {
      errorNotifications(response)
    } else {
      successNotification({ message: `${collection} update started!` })
      watchUpdates({ collection }) // start watching
    }
    setUpdatedCollections(false) // ask for up to date data
  }

"databaseSize": 189247488,
    "lastUpdate": "2021-11-02T03:44:39.692897Z",

When Manually Un/Re-Publishing - Able to Confirm Data Transfer.
 "brand": {
            "numberOfDocuments": 11,
            "isIndexing": false,
            "fieldDistribution": {
                "Address": 11,
                "PointOfContact": 11,
                "Social": 11,
                "avatar": 11,
                "awards": 11,
                "brandProfileMedia": 11,
                "description": 11,
                "id": 11,
                "name": 11,
                "phone": 11,
                "products": 11,
                "salons": 11,
                "slogan": 11,
                "slug": 11,
                "url": 11
            }
        },

Thanks @jbelke I'm looking into this :)

Hey @jbelke I think this PR solves the issue: https://github.com/meilisearch/strapi-plugin-meilisearch/pull/287
I tried it locally and it works.

Hey @jbelke I think this PR solves the issue: #287
I tried it locally and it works.

Great! Looking forward to the PR being merged.

fixed be #288 Could you confirm @jbelke ?

Considered fixed!


Workflow changes

There's some things that might help automate the workflow of merging things into this repository, e.g. adding bors-ng to streamline merging once tests pass and enabling merge commits. I can't do this as I'm not admin.
Is this something you're okay with doing @rafaelcaricio?

You decide! It sounds good to me.


Low performance for VM2VM in v2015.08

We are using the following script for VM2VM performance evaluation:
local VhostUser = require("apps.vhost.vhost_user").VhostUser

local c = config.new()
config.app(c, "vh1", VhostUser, {socket_path="/tmp/vh1.sock", is_server=false})
config.app(c, "vh2", VhostUser, {socket_path="/tmp/vh2.sock", is_server=false})

config.link(c, "vh1.tx -> vh2.rx")
config.link(c, "vh2.tx -> vh1.rx")

engine.configure(c)
engine.main()

This script is run using snsh and attached to the NUMA node where the 2 VMs are running.
Inside the VMs we use a packet generator based on DPDK from here http://dpdk.org/browse/apps/pktgen-dpdk/ The performance test is to send 64byte UDP packets from one VM and measure the received ones using pktgen-dpdk in the other VM.

+-----+---------+     +-----+---------+
|     |         |     |     |         |
|VM1  | PKTGEN  |     |VM2  | PKTGEN  |
|     |         |     |     |         |
|     +---------+     |     +---------+
|     |         |     |     |         |
|     |  DPDK   |     |     |  DPDK   |
|     |         |     |     |         |
+-----+-----+---+     +-----+---+-----+
            |                   |
            |                   |
+-----------v-------------------v-----+
|                                     |
|           SnabbSwitch               |
|                                     |
|       snsh vhost2vhost.lua          |
|                                     |
+-------------------------------------+

The test are run on a E5-2637 v3 @ 3.50GHz dual socket machine.
When using v2015.07.1 we are seeing about 12.7 Mpps passed to the receiving machine.
With the latest master aka v2015.08 the same performance test shows 9.3 Mpps.

Great catch!
@eugeneia has recently added a performance regression on the CI that I would expect to catch this. It is running the classic DPDK packet forwarding benchmark (loadgen->snabbnfv->vm->snabbnfv). However, this test may have been enabled for the first time after the regression was introduced.
@eugeneia can your regression test detect the performance difference between v2015.07 and v2015.08? That would be very cool :-).

@eugeneia also is there a simple instruction for how others (e.g. me) can run that test case manually?

@lukego I assume you want to test src/program/snabbnfv/packetblaster_bench.sh (which is self documenting). Since that does not exist in v2015.07 my approach would be to backport #547 into v2015.07.

which is self documenting

Point taken :-).
The challenge to quick testing now is that I am on nixos and test-env assumes that quite a lot of diverse software is installed and available e.g. everything needed to compile QEMU. I suppose that this is the problem that the Dockerized test environment will solve.

In the near future this kind of problem should be the perfect use case for Docker based testing (#588). If the test environment is based on Docker then it should be a one-liner for anybody to run and reproduce the problem. I would just type docker run vosys/vm2vmbench and the rest is automatic.
That would be wonderful because otherwise it can take anything from an hour to a day to setup a test environment by hand when dealing with complex software e.g. installing DPDK inside the guests and so on.

I did some bisecting and I found which commit causes this low perfromance in our test scenario.
This is the commit after which the performance drops down:
7063865 Revert "Merge branch 'restore-snabbtop' into next"
This is really strange as this commit only remove some code related to statistics in the app.lua.
This is the last known good commit:
8ecc0c0 ipv6:pton: Fix error on invalid address
Any hints how to profile with snsh will be useful.

@nnikolaev-virtualopensystems Great detective work. I also don't immediately see why this specific commit would be the issue. Let me braindump a little.
There has previously been a performance regression related to snabbtop that we tried to resolve with #568. That was an extremely hard to profile issue: we were accessing many counters on 4096-byte aligned addresses and CPU cache could not handle that. The backstory of finding that problem is on #558.
Our solution was to use a double-buffer to access the counters less frequently. However, we still write them once every 100 breathe() loops and maybe that is too often. Could be worth changing app.breathe() to call counter.commit() less often e.g. every 10000 loops and see if that affects the results.
@eugeneia has added a standard DPDK-based packet forwarding benchmark on the master branch now. I have tried to reproduce the issue with that but haven't seen it yet (but can be that I am making a git mistake when backporting it onto 8ecc0c0). It would be very useful if we could reproduce the problem using that test since it is the easiest setup for multiple people to run and also it is running automatically on the CI. If you can see the problem using that benchmark, or an adaptation of that benchmark to be more like your environment (one-way packet send instead of two-way), then that could help me to reproduce it over here (just one idea).
(Hopefully once we move our test environments over to Docker images it will be easier to run the test with an arbitrary version of the software. Currently the test scripts and the code are living in the same directory tree and it is not so easy to run a new test with older software from before the test was added. I know, I know, I am expecting Docker to solve all of our problems at once, we will see :))
Here is how that test runs:
$ sudo TESTPCI0=0000:01:00.0 TESTPCI1=0000:01:00.1 TELNET_PORT=5000 program/snabbnfv/packetblaster_bench.sh
...
load: time: 1.00s  fps: 4,756,445 fpGbps: 2.778 fpb: 218 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,935,922 fpGbps: 5.803 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,862,646 fpGbps: 5.760 fpb: 255 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,944,067 fpGbps: 5.807 fpb: 255 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,957,900 fpGbps: 5.815 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,932,911 fpGbps: 5.801 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,922,261 fpGbps: 5.795 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,924,639 fpGbps: 5.796 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,919,184 fpGbps: 5.793 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,941,215 fpGbps: 5.806 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,945,911 fpGbps: 5.808 fpb: 255 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,508,429 fpGbps: 5.553 fpb: 248 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,984,065 fpGbps: 5.831 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,972,250 fpGbps: 5.824 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,949,254 fpGbps: 5.810 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,983,303 fpGbps: 5.830 fpb: 256 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,204,803 fpGbps: 5.376 fpb: 242 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,355,580 fpGbps: 5.464 fpb: 242 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,196,086 fpGbps: 5.371 fpb: 249 bpp: 64   sleep: 0   us
load: time: 1.00s  fps: 9,951,668 fpGbps: 5.812 fpb: 256 bpp: 64   sleep: 0   us
load: time: 0.91s  fps: 9,988,466 fpGbps: 5.833 fpb: 256 bpp: 64   sleep: 0   us

Processed 100.0 million packets in 20.40 seconds<PHONE_NUMBER> bytes; 2.51 Gbps)
Made 890,951 breaths: 112.24 packets per breath; 22.90us per breath
Rate(Mpps):     4.901

For profiling with snsh here is one idea:
snabb snsh -jp=z -p snabbnfv ...

which should tell you how time is spent in "zones" (z) which is separately for each app i.e. which part is getting slower of vhost-user vs intel_app vs engine.
You could also profile with perf stat e.g.
perf stat -e instructions,cycles,L1-dcache-misses,cache-misses,dTLB-load-misses,dTLB-store-misses,iTLB-load-misses

which may lead to some interesting information. For example if you see many more instructions for the same work then it might point to the JIT. If you see lower instructions per cycle it might point to the CPU.
I have been using a personal blog lately to explore low-level profiling issues around both LuaJIT and Xeon E5. I am starting at the most basic and working my way up to Snabb Switch. If you are interested there are four entries starting with lukego/blog#5. Could make LuaJIT slightly less mysterious :).

I am closing this because it seems to be a stale issue. Please reopen/object in case this is still relevant.


fixed issue #9189

Issue Reference
fixed issue #9189

@shivendra-webkul
Checked and Found that the issue is not fixed. Please find the screenshot here for reference.

@shivendra-webkul
Checked and Found that the issue has been fixed. Please check the screenshots for reference.
RTL View - https://prnt.sc/if9VSC0SpXqX
LTR View - https://prnt.sc/vYu3OXcFcDAx


Disable CSRF protection on API

(Hit into this myself while trying to test out an endpoint with curl)

See, for example:

https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/request_forgery_protection.rb#L15
https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/graphql_controller.rb#L30
https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/sessions_controller.rb#L52

for other examples of the difference between UI and API auth


Resources slated for deletion are not actually removed

https://github.com/user-attachments/assets/1ddc93f9-3de0-4372-acd6-0846f0fff004

I see now that the delete jobs are slated for Celery but remain pending
closing in favor of more general Celery async improvements, for instance
https://github.com/hydroshare/hydroshare/issues/5597


Purpose of torch.set_num_threads(1)

Hi, thanks for releasing a decent VAD model!
While looking through examples, I saw multiple times usage of: torch.set_num_threads(1)
I haven't observed any performance difference with and without num_threads = 1 both for jit and onnx models.
Curious, what's the intended purpose of that statement?

While looking through examples, I saw multiple times usage of: torch.set_num_threads(1)

This is to indicate that the VAD needs only one CPU thread and does not consumer much compute.

I haven't observed any performance difference with and without num_threads = 1 both for jit and onnx models.

This is correct. There should be very little, if any, difference. But when you do not set it, PyTorch eats all CPU threads by default.
Also note that ONNX has a different way of setting its number of threads.


Class literals for existential value types do not match the JVM

The test RegressionTest.should_support_class_literals_for_existential_value_types_issue_218 does not pass on the JVM. The code of the test is:
  @Test def should_support_class_literals_for_existential_value_types_issue_218(): Unit = {
    assertEquals("org.scalajs.testsuite.compiler.RegressionTest$Bug218Foo",
        scala.reflect.classTag[Bug218Foo[_]].toString)
  }

and the error is:
[error] Test org.scalajs.testsuite.compiler.RegressionTest.should_support_class_literals_for_existential_value_types_issue_218 failed: org.junit.ComparisonFailure: expected:<[org.scalajs.testsuite.compiler.RegressionTest$Bug218Foo]> but was:<[Object]>, took 0.001 sec
[error]     at org.scalajs.testsuite.compiler.RegressionTest.should_support_class_literals_for_existential_value_types_issue_218(RegressionTest.scala:148)
[error]     ...

This was initially implemented in #245 to fix the issue #218. Somehow we became smarter than Scala/JVM, since we can actually give a precise class literal for the value class, whereas Scala/JVM gives Object. We should be bug-compatible, though.

@sjrd the PR for this just got merged (in case you didn't notice); should this get closed? Thanks!

Will make sure whether we need to do anything (like adapting a test) when I'm back from vacation.

Fixed in e259939196706f9a7d69e909db15e504b25f379d.


Ensure SQS property WaitTimeSeconds is an integer

This just ensure that the WaitTimeSeconds property is in integer.

Coverage decreased (-0.1%) to 93.333% when pulling bd2ff89bf127315f398f49c734cdb25eaf96bfdf on scawley-r7:master into cda553abfb84ca575f9fe24d546fbd2117f302a8 on spulec:master.

Coverage decreased (-0.1%) to 93.333% when pulling bd2ff89bf127315f398f49c734cdb25eaf96bfdf on scawley-r7:master into cda553abfb84ca575f9fe24d546fbd2117f302a8 on spulec:master.

Thanks!


Announcement of release 1.0

Links to ontology from wiki, etc.
Internal and/or public announcement.

Public announcement must be done sometime soon but perhaps not critical for the April 22 milestone.

Can be done in coordination with Michelle, but she doesn't have a github username.

@rjyounes : are we waiting to post the ontology to biblioteko/-o.org before announcing OR are we linking to the github repo? I was assuming the former but if that definitely won't be up and running for next week, I can send the announcement to various places once we decide on the hyphen issue... since I'll also share the pattern docs.

@rjyounes : just spoke with Michelle. She thinks we should wait until after the community & partner meetings to make a public announcement as communication strategy is on the agenda -- so this will inform. Plus we won't have much opportunity to engage next week if/when comments come in.

@jak473  Sounds fine to me. I was just going to ask you about date on the ontology. Currently I've set it for 4/21 at midnight, so that means it will predate the announcement. I think that's okay; would you agree?

@jak473 Does this mean we will not put links to ontology up on ld4l.org before the meeting?

@rjyounes : I don't think that is an issue; I just mean that we won't post its existence on listservs, etc., until we have a strategy. Cross-linking between wikis and websites we own works just fine (to me at least)


Implement Timing Channels in Engine

[x] This contribution adheres to CONTRIBUTING.md.

What does this Pull Request accomplish?

Create mechanism for storing the Incoming Channel index offset and current execution time per execution unit to report execution duration to the appropriate timing channel

Add accessors to the Execution Unit implementation for Timing Offset and Current Execution Time
Add methods and overrides to the Execution Unit interface and implementation for Update Current Execution Time and Write Execution Time
Set the offset to a non-negative (valid) value when creating each execution unit
Use the offset to replace the array element of the incoming channel data array per execution unit

Create two new sub VIs for use in the engine (Timed Read from Hardware and Timed Write to Hardware)

These call into the respective existing VIs and use the microsecond tick count VI to calculate execution time
Because the Write VI does not modify the incoming data, store the Current Execution Time
In the Read VI, add its duration and the current execution time, then update the value in the data array

Add system test to deploy with Timing Channels section added

Why should this Pull Request be merged?
Give users an optional set of timing channels to analyze per-execution unit execution duration to determine custom device performance implications on their system.
What testing has been done?

Deployed with various system configurations to see execution time change as expected
Ran test suite

Running into problems while running all of the system tests together. Will investigate tomorrow.


Megadraft internal i18next instance messes up App's one

Since 0.5.0, importing Megadraft causes an override of my app's i18next instance. In other words, in the moment the component with Megadraft import is loaded, all others translations show only their key values.
Can be i18next disabled (or removed) from Megadraft, and provide custom translations as props, like other component libraries handle i18n. Here is an example:
const i18n = {
  linkPlaceholder: "Type the link and press enter",
  linkInvalid: "Invalid Link",
  pluginMissing:  (type) => `Can't show plugin, component "${type}" not found.`,
  blockList: 'Block List'
}

<MegadraftEditor
  editorState={editorState}
  onChange={this.onChange}
  i18n={i18n}
  />

Removing the dependency can give Megadraft more flexibility, as users will be able integrate the i18n library they want, without bundling a library they won't use. It will also mean that Megadraft won't need to support a set of languages (if they need something else, developers can take care of that).
Another option is to be able to provide a custom i18next instance that will be used instead of Megadraft's one:
import i18n from '../my/own/i18n' // <- this will also include Megadraft translations

<MegadraftEditor
  editorState={editorState}
  onChange={this.onChange}
  i18n={i18n}
  />

@eduludi, yes, Megadraft should not take control and override i18next, your first suggestion looks good to me.

#219 fixes it, so I'm closing this issue.


Template Artifact Builds

Is your feature request related to a problem? Please describe.
Having to fill in all of the details for a point is going to get really old. It would be nice if I could save a point under a template, then just assign that spec to a point/points.
Describe the solution you'd like
If we could have an option on a specific point to the effect of save as template that is one way this could be done. Then users could save that template to the system for everyone to use.
Describe alternatives you've considered
Originally I thought about tying it to a specific culture... And we probably should do this somehow seeing as a "culture" is essentially a template for a point's attributes. Perhaps this is where the templates could live and be managed from. As part of the culture page. This is a distant feature, should meet on it when the time comes.

Is your feature request related to a problem? Please describe. Having to fill in all of the details for a point is going to get really old. It would be nice if I could save a point under a template, then just assign that spec to a point/points.
Describe the solution you'd like If we could have an option on a specific point to the effect of save as template that is one way this could be done. Then users could save that template to the system for everyone to use.
Describe alternatives you've considered Originally I thought about tying it to a specific culture... And we probably should do this somehow seeing as a "culture" is essentially a template for a point's attributes. Perhaps this is where the templates could live and be managed from. As part of the culture page. This is a distant feature, should meet on it when the time comes.

Yes, agreed. Culture/point types should be saved templates, and all the main types found in Sask, with the appropriate age/date ranges should be pre-saved as templates for our users. Also, I'm still hoping to get some line drawings of the main types, to add to the templates, to help folks recognize which they have.


issue Saving

Error Code
Error: Too Many Requests

Error: Too Many Requests
    at Request.<anonymous> (https://homebrewery.naturalcrit.com/homebrew/bundle.js:78380:21)
    at Emitter.emit (https://homebrewery.naturalcrit.com/homebrew/bundle.js:1954:20)
    at XMLHttpRequest.<anonymous> (https://homebrewery.naturalcrit.com/homebrew/bundle.js:78674:10)
{
  "status": 429,
  "method": "PUT",
  "url": "/api/update/T8E1pB7gcxFl"
}

### Your description of what happened:

got this error again

Merging to mega-issue #3770.