Script contains multiple redeclaration cases

Localize.swift contains multiple redeclaration cases of some variables like sanitizeFiles or singleLanguage.
All these errors can be seen on the attached screenshot.
On the other hand the version of Localize.swift that is provided in the Starter Project compiles and does not contain any redeclaration cases. Although this version of the script looks older and outdated.
Can you please provide some more info why this is the case and how should the script be fixed into a working state.

@crtgregoric I think something broke with the latest commit
https://github.com/freshOS/Localize/commit/0221deed4ddd68d13becbdae45015f69f4be00da
@NikKovIos  Could you please take look?
Not sure the latest commit is production ready^^
In the meantime You can checkout 8f841bb973aadbfc36e0bdde1f2c74003401c7da that is a working version.
#!/usr/bin/env xcrun --sdk macosx swift

import Foundation

// WHAT
// 1. Find Missing keys in other Localisation files
// 2. Find potentially untranslated keys
// 3. Find Duplicate keys
// 4. Find Unused keys and generate script to delete them all at once

/*
 Put your path here, example ->  Resources/Localizations/Languages
 */
let relativeLocalizableFolders = "/Resources/Languages"

/*
 This is the path of your source folder which will be used in searching
 for the localization keys you actually use in your project
 */
let relativeSourceFolder = ""

/*
 Those are the regex patterns to recognize localizations.
 */
let patterns = [
    "NSLocalized(Format)?String\\(\\s*@?\"([\\w\\.]+)\"", // Swift and Objc Native
    "Localizations\\.((?:[A-Z]{1}[a-z]*[A-z]*)*(?:\\.[A-Z]{1}[a-z]*[A-z]*)*)", // Laurine Calls
    "L10n.tr\\(key: \"(\\w+)\"", // SwiftGen generation
    "\"(.*)\".localized" // "exampleKey".localized
]

/*
 Those are the keys you don't want to be recognized as "unused"
 For instance, Keys that you concatenate will not be detected by the parsing
 so you want to add them here in order not to create false positives :)
 */
let ignoredFromUnusedKeys = [String]()
/* example
let ignoredFromUnusedKeys = [
    "NotificationNoOne",
    "NotificationCommentPhoto",
    "NotificationCommentHisPhoto",
    "NotificationCommentHerPhoto"
]
*/

let masterLanguage = "en"

/*
 Sanitizing files will remove comments, empty lines and order your keys alphabetically.
 */
let sanitizeFiles = true

/*
 Determines if there are multiple localizations or not.
 */
let singleLanguage = false

// MARK: - End Of Configurable Section

// Detect list of supported languages automatically
func listSupportedLanguages() -> [String] {
    var sl = [String]()
    let path = FileManager.default.currentDirectoryPath + relativeLocalizableFolders
    if !FileManager.default.fileExists(atPath: path) {
        print("Invalid configuration: \(path) does not exist.")
        exit(1)
    }
    let enumerator: FileManager.DirectoryEnumerator? = FileManager.default.enumerator(atPath: path)
    let extensionName = "lproj"
    print("Found these languages:")
    while let element = enumerator?.nextObject() as? String {
        if element.hasSuffix(extensionName) {
            print(element)
            let name = element.replacingOccurrences(of: ".\(extensionName)", with: "")
            sl.append(name)
        }
    }
    return sl
}

let supportedLanguages = listSupportedLanguages()
var ignoredFromSameTranslation = [String:[String]]()
let path = FileManager.default.currentDirectoryPath + relativeLocalizableFolders
var numberOfWarnings = 0
var numberOfErrors = 0

struct LocalizationFiles {
    var name = ""
    var keyValue = [String:String]()
    var linesNumbers = [String:Int]()

    init(name: String) {
        self.name = name
        process()
    }

    mutating func process() {
        if sanitizeFiles {
            removeCommentsFromFile()
            removeEmptyLinesFromFile()
            sortLinesAlphabetically()
        }
        let location = singleLanguage ? "\(path)/Localizable.strings" : "\(path)/\(name).lproj/Localizable.strings"
        if let string = try? String(contentsOfFile: location, encoding: .utf8) {
            let lines =  string.components(separatedBy: CharacterSet.newlines)
            keyValue = [String:String]()
            let pattern = "\"(.*)\" = \"(.+)\";"
            let regex = try? NSRegularExpression(pattern: pattern, options: [])
            var ignoredTranslation = [String]()

            for (lineNumber, line) in lines.enumerated() {
                let range = NSRange(location:0, length:(line as NSString).length)

                // Ignored pattern
                let ignoredPattern = "\"(.*)\" = \"(.+)\"; *\\/\\/ *ignore-same-translation-warning"
                let ignoredRegex = try? NSRegularExpression(pattern: ignoredPattern, options: [])
                if let ignoredMatch = ignoredRegex?.firstMatch(in:line,
                                                                       options: [],
                                                                       range: range) {
                    let key = (line as NSString).substring(with: ignoredMatch.range(at:1))
                    ignoredTranslation.append(key)
                }
                if let firstMatch = regex?.firstMatch(in: line, options: [], range: range) {
                    let key = (line as NSString).substring(with: firstMatch.range(at:1))
                    let value = (line as NSString).substring(with: firstMatch.range(at:2))
                    if let _ =  keyValue[key] {
                        let str = "\(path)/\(name).lproj"
                        + "/Localizable.strings:\(linesNumbers[key]!): "
                        + "error: [Redundance] \"\(key)\" "
                        + "is redundant in \(name.uppercased()) file"
                        print(str)
                        numberOfErrors += 1
                    } else {
                        keyValue[key] = value
                        linesNumbers[key] = lineNumber+1
                    }
                }
            }
            print(ignoredFromSameTranslation)
            ignoredFromSameTranslation[name] = ignoredTranslation
        }
    }

    func rebuildFileString(from lines: [String]) -> String {
        return lines.reduce("") { (r: String, s: String) -> String in
            return (r == "") ? (r + s) : (r + "\n" + s)
        }
    }

    func removeEmptyLinesFromFile() {
        let location = "\(path)/\(name).lproj/Localizable.strings"
        if let string = try? String(contentsOfFile: location, encoding: .utf8) {
            var lines = string.components(separatedBy: CharacterSet.newlines)
            lines = lines.filter { $0.trimmingCharacters(in: CharacterSet.whitespaces) != "" }
            let s = rebuildFileString(from: lines)
            try? s.write(toFile:location, atomically:false, encoding:String.Encoding.utf8)
        }
    }

    func removeCommentsFromFile() {
        let location = "\(path)/\(name).lproj/Localizable.strings"
        if let string = try? String(contentsOfFile: location, encoding: .utf8) {
            var lines = string.components(separatedBy: CharacterSet.newlines)
            lines = lines.filter { !$0.hasPrefix("//") }
            let s = rebuildFileString(from: lines)
            try? s.write(toFile:location, atomically:false, encoding:String.Encoding.utf8)
        }
    }

    func sortLinesAlphabetically() {
        let location = "\(path)/\(name).lproj/Localizable.strings"
        if let string = try? String(contentsOfFile: location, encoding: .utf8) {
            let lines = string.components(separatedBy: CharacterSet.newlines)

            var s = ""
            for (i,l) in sortAlphabetically(lines).enumerated() {
                s += l
                if (i != lines.count - 1) {
                    s += "\n"
                }
            }
            try? s.write(toFile:location, atomically:false, encoding:String.Encoding.utf8)
        }
    }

    func removeEmptyLinesFromLines(_ lines:[String]) -> [String] {
        return lines.filter { $0.trimmingCharacters(in: CharacterSet.whitespaces) != "" }
    }

    func sortAlphabetically(_ lines:[String]) -> [String] {
        return lines.sorted()
    }
}

// MARK: - Load Localisation Files in memory

let masterLocalizationfile = LocalizationFiles(name: masterLanguage)
let localizationFiles = supportedLanguages
    .filter { $0 != masterLanguage }
    .map { LocalizationFiles(name: $0) }

// MARK: - Detect Unused Keys

let sourcesPath = FileManager.default.currentDirectoryPath + relativeSourceFolder
let fileManager = FileManager.default
let enumerator = fileManager.enumerator(atPath:sourcesPath)
var localizedStrings = [String]()
while let swiftFileLocation = enumerator?.nextObject() as? String {
    // checks the extension // TODO OBJC?
    if swiftFileLocation.hasSuffix(".swift") ||  swiftFileLocation.hasSuffix(".m") || swiftFileLocation.hasSuffix(".mm") {
        let location = "\(sourcesPath)/\(swiftFileLocation)"
        if let string = try? String(contentsOfFile: location, encoding: .utf8) {
            for p in patterns {
                let regex = try? NSRegularExpression(pattern: p, options: [])
                let range = NSRange(location:0, length:(string as NSString).length) //Obj c wa
                regex?.enumerateMatches(in: string,
                                                options: [],
                                                range: range,
                                                using: { (result, _, _) in
                    if let r = result {
                        let value = (string as NSString).substring(with:r.range(at:r.numberOfRanges-1))
                        localizedStrings.append(value)
                    }
                })
            }
        }
    }
}

var masterKeys = Set(masterLocalizationfile.keyValue.keys)
let usedKeys = Set(localizedStrings)
let ignored = Set(ignoredFromUnusedKeys)
let unused = masterKeys.subtracting(usedKeys).subtracting(ignored)

// Here generate Xcode regex Find and replace script to remove dead keys all at once!
var replaceCommand = "\"("
var counter = 0
for v in unused {
    var str = "\(path)/\(masterLocalizationfile.name).lproj/Localizable.strings:\(masterLocalizationfile.linesNumbers[v]!): "
    str += "error: [Unused Key] \"\(v)\" is never used"
    print(str)
    numberOfErrors += 1
    if counter != 0 {
        replaceCommand += "|"
    }
    replaceCommand += v
    if counter == unused.count-1 {
        replaceCommand += ")\" = \".*\";"
    }
    counter += 1
}

print(replaceCommand)

// MARK: - Compare each translation file against master (en)

for file in localizationFiles {
    for k in masterLocalizationfile.keyValue.keys {
        if let v = file.keyValue[k] {
            if v == masterLocalizationfile.keyValue[k] {
                if !ignoredFromSameTranslation[file.name]!.contains(k) {
                    let str = "\(path)/\(file.name).lproj/Localizable.strings"
                    + ":\(file.linesNumbers[k]!): "
                    + "warning: [Potentialy Untranslated] \"\(k)\""
                    + "in \(file.name.uppercased()) file doesn't seem to be localized"
                    print(str)
                    numberOfWarnings += 1
                }
            }
        } else {
            var str = "\(path)/\(file.name).lproj/Localizable.strings:\(masterLocalizationfile.linesNumbers[k]!): "
            str += "error: [Missing] \"\(k)\" missing form \(file.name.uppercased()) file"
            print(str)
            numberOfErrors += 1
        }
    }
}

print("Number of warnings : \(numberOfWarnings)")
print("Number of errors : \(numberOfErrors)")

if numberOfErrors > 0 {
    exit(1)
}

Wow, sure, sorry for that, i gonna fix it if this is the bug cos of me!

No worries buddy :) The last commit is pretty huge, was it pushed un purpose ?

That was a mistake i think. Gonna watch tomorrow)

@s4cha Thanks for the working version.

Check this commit please https://github.com/freshOS/Localize/commit/1a7a5415e37a03509a80c5d4510131302feceae4 Are the errors disappears?


Powertoy Run should work in every drive

Description of the new feature / enhancement
My PowerToys run seem to work only for Drive C, as when I search a folder in my Drive F, it show nothing.
Scenario when this would be used?

Supporting information
No response

@monykappa
Wat do you searching for? If you only search for files this documentation might help you: https://learn.microsoft.com/en-us/windows/powertoys/run#windows-search-settings
/needinfo

@htcfreek
Thank you, it is working right now 🙏!!

/helped


Increase OEV gateway memory

We've been discussing plans to increase the OEV gateway memory to reuse the OEV gateway implementation in https://github.com/api3dao/oev-auctioneer/issues/6. I've done some experiments and the cold start varies depending on the gateway memory. This is because the lambda gets CPU proportionally to the memory (and higher memory means higher CPU which decreases the work at cold start).
I am opening a PR which changes the GW memory to 2048 (not configurable).

I was waiting for our discussion in Slack to end, which I think did and I think this is good to go.


A typo fix and a grammar suggestion

There was already a pull request for the typo. So, I merged only one commit of your pull reqeuest manually. Thanks.


Proposal: Add Homepage, License and Author fields to Plugin

I propose new three fields for Plugin.

spec.homepage:  Optional project homepage, only for documentation purposes

Motivation: When I find bugs in a plugin, I’d like to know the repository of the plugin easily in order to send an issue.

spec.license: Optional license such as Apache-2.0 or MIT, only for documentation purposes

Motivation: When I find a plugin as it seems convenient, I’d like to know the license of the plugin easily.

spec.author: Optional project author, only for documentation purposes

Motivation: When I find an awesome plugin, I’d like to know the author of the plugin easily.

Along with adding the new fields, krew info command will be changed to show homepage, license, and author as follows:
$ kubectl krew info krew
NAME: krew
URI: https://storage.googleapis.com/krew/v0.2.1/krew.tar.gz
SHA256: dc2f2e1ec8a0acb6f3e23580d4a8b38c44823e948c40342e13ff6e8e12edb15a
VERSION: v0.2.1
HOMEPAGE: https://github.com/GoogleContainerTools/krew
LICENSE: Apache-2.0
AUTHOR: https://github.com/ahmetb
CAVEATS:
\
 |  krew is now installed! To start using kubectl plugins, you need to add
 |  krew's installation directory to your PATH:
 |    * macOS/Linux:
 |      - Add the following to your ~/.bashrc or ~/.zshrc:
 |          export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"
 |      - Restart your shell.
 |    * Windows: Add %USERPROFILE%\.krew\bin to your PATH environment variable
 |  Run "kubectl krew" to list krew commands and get help.
 |  You can find documentation at https://github.com/GoogleContainerTools/krew.
/

The below is krew.yaml which is added the new three fields:
diff --git a/plugins/krew.yaml b/plugins/krew.yaml
index dca2408..e811ac0 100644
--- a/plugins/krew.yaml
+++ b/plugins/krew.yaml
@@ -19,6 +19,9 @@ metadata:
 spec:
   version: "v0.2.1"
   shortDescription: Package manager for kubectl plugins.
+  homepage: https://github.com/GoogleContainerTools/krew
+  license: Apache-2.0
+  author: https://github.com/ahmetb
   caveats: |
     krew is now installed! To start using kubectl plugins, you need to add
     krew's installation directory to your PATH:

https://github.com/superbrothers/krew/tree/homepage-license-author is PoC of this proposal.

Thanks for the proposal. I think in the immediate term we need a link that goes to documentation or project home page so that we can point users to the documentation that way. So +1 for that.
As for author, in many cases there will be multiple, so I think we should leave the author out.
Similarly with license, I am erring on the side of not including it as almost any package manager (unless it's a language module manager like npm, pip etc), won't tell you what’s the involved license.
So I'm okay with adding a homepage field and proactively update existing plugins. 👍

Thank you for your feedback. I understand your opinion. So I will create a PR which adds a homepage field. :blush:


setup.py lacks  roman  dependency

lacks the roman dep I think ?
@ setup.py:12 @ setup(name='termpdf.py',
<EMAIL_ADDRESS>      url='https://github.com/dsanson/termpdf.py',
      scripts=['termpdf.py'],
      requires=['PyMuPDF', 'pyperclip']
      requires=['PyMuPDF', 'pyperclip', 'roman']
     )

For me this was missing in the requirements.txt:
roman
pdfrw
pagelabels

would it be possible to move (or duplicate) the requirements to setup.py please ? it' the best way to distribute/install packages https://packaging.python.org/discussions/install-requires-vs-requirements/

Sorry. Should have up to date requirements.txt and setup.py now.


don't play sound while the finger is down

admittedly it feels a bit different now, but it doesn't make sense to me why we play boops while the finger is on the screen

i dig it


LAB_04L01_Implement_and_manage_AVD_profiles_ADDS

Module: 04
Lab/Demo: 01
Task: 01
Step: 13
Description of issue
Related to #30
The only addition to the lab was a note after the installation steps saying installation is not required if the image already includes it.
This is not good enough. Students, when doing the labs, will read the instructions and then see the note.
Can this step not be reworded or an extra step added to Start | Run | appwiz.cpl and check for the presence of Microsoft FSLogix apps?

Or, as per @ningweinw's suggestion remove the step entirely given the VMs are marketplace images?

Pls submit a PR providing the wording you'd like to see being used
rgds
Marcin


Avoid race conditions between assign and initial

The initial execution of a assign statement is scheduled in the same time slot as an initial block.
There are a few tests that use an initial block to evaluate whether an assign statement was correctly executed. But this leads to an inherit race condition as the initial block might run before the assign has been executed.
Switch these tests to use a final block instead to avoid the race condition.

Hm, looks like Odin does not support final. But then again Odin does not support simulation and these are all simulation tests. Maybe just disable simulation tests for Odin?


A question about getObjectPose.m

#Hello,I have a question about this:
In line 107 to 124:
% Do 3D background subtraction
% pcwrite(pointCloud(objSegmPts'),fullfile(visPath,sprintf('vis.seg.%s',objName)),'PLYFormat','binary');
% pcwrite(backgroundPointCloud,fullfile(visPath,sprintf('vis.bg.%s',objName)),'PLYFormat','binary');
if useGPU
    [indicesNN,distsNN] = multiQueryKNNSearchImplGPU(backgroundPointCloud,objSegmPts');
else
    [indicesNN,distsNN] = multiQueryKNNSearchImpl(backgroundPointCloud,objSegmPts',1);
end
objSegmPts(:,find(sqrt(distsNN) < 0.005)) = [];
if strcmp(sceneData.env,'shelf')
  objSegmPtsBg = extBin2Bg(1:3,1:3) * objSegmPts + repmat(extBin2Bg(1:3,4) + [0;0;0.01],1,size(objSegmPts,2));
  bgRot = vrrotvec2mat([0 1 0 -atan(0.025/0.20)]);
  objSegmPtsBgRot = bgRot(1:3,1:3) * objSegmPtsBg;
%   pcwrite(pointCloud(objSegmPtsBgRot'),fullfile(visPath,sprintf('vis.objTestRot.%s',objName)),'PLYFormat','binary');
%   pcwrite(pointCloud(objSegmPts'),fullfile(visPath,sprintf('vis.objTestPre.%s',objName)),'PLYFormat','binary');
  objSegmPts(:,find(objSegmPtsBg(3,:) < -0.025 | objSegmPtsBgRot(3,:) < 0)) = [];
%   pcwrite(pointCloud(objSegmPts'),fullfile(visPath,sprintf('vis.objTestPost.%s',objName)),'PLYFormat','binary');
end

In line 115,There may be a missing line here:
objSegmConf(:,find(sqrt(distsNN) < 0.005)) = [];
In line 117 and 118:
  objSegmPtsBg = extBin2Bg(1:3,1:3) * objSegmPts + repmat(extBin2Bg(1:3,4) + [0;0;0.01],1,size(objSegmPts,2));
  bgRot = vrrotvec2mat([0 1 0 -atan(0.025/0.20)]);

Why do I add 0.01 to the Z axis and how do I get 0.025 and 0.2. What is the purpose of doing this?
Please help solve this question, thanks!

For line 115, I think you are right -- that line should be added. I've put in a commit for it, thanks for the catch!
Each shelf bin in the Amazon Picking Challenge is not completely flat on the bottom. Half is flat, and another half has a small incline. The goal of line 117 and line 118 is to shift and rotate the point cloud of the shelf bin, so that it is easier to remove points (line 122) that are below the inclined part of the bin bottom.

Thanks a lot!


File orginization

Putting pages into ./pages and components into their respective features

can I do this one?


Automated cherry pick of #11940

Cherry pick of #11940 on release-7.7.
/cc  @koox00
NONE

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.

Trying to auto merge this PR.

Pull Request successfully merged
SHA: fad2fa843619fee9985074ea327f8363fef9ba47


How to use code interpreter to save gpt generated documents such as csv files to local disk

I have a requirement to generate an excel table according to some known data and save the table to the local disk, I can not save to the local disk, it is always saved in the directory（ /mnt/data/XXX.csv）, I guess it should be the directory of the virtual environment. I have an idea to execute the code generated by the code interpreter locally, but I don't know how to execute it or save the file. Is there any other simpler solution?

A simple solution I can see is to not use docker (set use_docker to False in the code_execution_config argument ). In that case, the program saves generated files to a dir on your local machine. You can either use the default work_dir or specify a specific local directory (via the work_dir field of the code_execution_config argument) to save the generated file to the local disk.
Check code_execution_config in this SDK doc: https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent
"work_dir (Optional, str): The working directory for the code execution. If None, a default working directory will be used. The default working directory is the "extensions" directory under "path_to_autogen"."

A simple solution I can see is to not use docker (set use_docker to False in the code_execution_config argument ). In that case, the program saves generated files to a dir on your local machine. You can either use the default work_dir or specify a specific local directory (via the work_dir field of the code_execution_config argument) to save the generated file to the local disk.
Check code_execution_config in this SDK doc: https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent
"work_dir (Optional, str): The working directory for the code execution. If None, a default working directory will be used. The default working directory is the "extensions" directory under "path_to_autogen"."
If I call code interpreter in openai's assistant, how do I guarantee to run python code locally and save files such as scv locally

@lihui123456 can you share a snippet of your code? Which agent class are you using?

@gagb here is my code:
llm_config = {
"config_list": config_list,
"assistant_id": assistant_id,
"tools": [
{
"type": "code_interpreter"
}
],
}
gpt_assistant = GPTAssistantAgent(
name="assistant",
instructions="你是一个代码解释器助手，如果需要使用python代码请用代码来解决问题",
llm_config=llm_config)
user_proxy = UserProxyAgent(
name="user_proxy",
is_termination_msg=lambda msg: "TERMINATE" in msg["content"],
code_execution_config={
"work_dir": "coding",
"use_docker": False,  # set to True or image name like "python:3" to use docker
},
human_input_mode="NEVER"
)
user_proxy.initiate_chat(
gpt_assistant,
message="随机生成一个csv文件,并保存在本地",
)
If I execute the above code, the path of the generated CSV file is /mnt/data/XXX.csv, as if in a virtual environment, how can I use the file generated by gpt's assistant to locally or download the file generated in the virtual environment? thank you.

That's what I thought. This is about GPTAssistantAgent -- this was on the roadmap (#602) but never addressed. I just created an issue for it: https://github.com/microsoft/autogen/issues/916

Closing in lieu of #916


Project messages: fix my earlier fix

I had the checkbox still outside the .alert div.

coverage: 94.454%. remained the same
when pulling d343295720b920ea7683565b49a1ae8132dc93f2 on nimmo-fix-project-messages
into 7e43abab82824722f16c4c4d8ad54a1ba0cf3cf7 on main.


[Feature Request] Would be nice to have Skip/Close option in StepWidgetBuilder.useDefaultTheme

First of all thank you for the awesome package.
I have a small default feature request, it would be nice to have Skip/Close button in StepWidgetBuilder.useDefaultTheme, as most of the use cases will need a Skip button, which is better than creating a custom widgetBuilder for most of the users.
It should have these options:

StepWidgetBuilder.useDefaultTheme should have a parameter showSkip or showFinish
skipButtonTextBuilder to customize it's text
If it is the last index this should not be shown
Should have optional parameters to pass text style and button style e.g. skipButtonTextStyle, skipButtonStyle

Same thing can be done for a Previous button, so that out-of-the-box the library provides easy customizations

https://github.com/tal-tech/flutter_intro/blob/764cdff8c266f3207c50e6da78d8538050fdaca8/example/lib/main.dart#L155-L191
Hello, I am very glad you pointed out this problem. After careful consideration, I think that adding too many properties to useDefaultTheme will increase the difficulty of use. So I added a useAdvancedTheme method to customize the style more freely. The final display of results as shown in the picture below.

Thank you very much for the prompt update. The new  useAdvancedTheme is even better


Retrieving list of Subreddit emojis

The Subreddit model should include functions to access the emoji endpoints:
https://www.reddit.com/dev/api/#section_emoji
Users with the read scope should be able to access the list of emojis available on a given subreddit, and users with the structuredstyles scopes should be able to upload and delete emojis.

Pretty sure this repo is unmaintained.


flutter test cannot run offline

Steps to reproduce:

Disconnect from internet
Run any flutter test

Failed to load "/Users/dnfield/src/flutter/flutter/packages/flutter/test/widgets/image_test.dart": Failed host lookup: 'flutter-gold.skia.org' (OS Error: nodename nor servname provided, or not known, errno = 8)
package:flutter_goldens_client/skia_client.dart 318:46                                             SkiaGoldClient.getExpectations.<fn>
===== asynchronous gap ===========================
dart:_http                                                                                HttpOverrides.runWithHttpOverrides
package:flutter_goldens_client/skia_client.dart 312:28                                             SkiaGoldClient.getExpectations
package:flutter_goldens/flutter_goldens.dart 479:21                                                FlutterLocalFileComparator.fromDefaultComparator
package:flutter_goldens/flutter_goldens.dart 40:61                                                 main
/var/folders/ly/j71v2zl54n74gp72pw876xv800k69x/T/flutter_test_listener.MCatGI/listener.dart 47:18  main.<fn>.<fn>
===== asynchronous gap ===========================
package:test_api                                                                                RemoteListener.start
/var/folders/ly/j71v2zl54n74gp72pw876xv800k69x/T/flutter_test_listener.MCatGI/listener.dart 17:25  serializeSuite
/var/folders/ly/j71v2zl54n74gp72pw876xv800k69x/T/flutter_test_listener.MCatGI/listener.dart 43:36  main

✖ loading /Users/dnfield/src/flutter/flutter/packages/flutter/test/widgets/image_test.dart
Exited (1)

This particular test is not a golden test (even though it's in image_test.dart).
/cc @Piinks

Huh! I thought I fixed this a while back. If baselines can’t be retrieved
for comparison it should just skip. Thanks for letting me know!
On Thu, Jan 9, 2020 at 8:12 AM Dan Field<EMAIL_ADDRESS>wrote:

Steps to reproduce:

Disconnect from internet
Run any flutter test

Failed to load "/Users/dnfield/src/flutter/flutter/packages/flutter/test/widgets/image_test.dart": Failed host lookup: 'flutter-gold.skia.org' (OS Error: nodename nor servname provided, or not known, errno = 8)
package:flutter_goldens_client/skia_client.dart 318:46                                             SkiaGoldClient.getExpectations.
===== asynchronous gap ===========================
dart:_http                                                                                HttpOverrides.runWithHttpOverrides
package:flutter_goldens_client/skia_client.dart 312:28                                             SkiaGoldClient.getExpectations
package:flutter_goldens/flutter_goldens.dart 479:21                                                FlutterLocalFileComparator.fromDefaultComparator
package:flutter_goldens/flutter_goldens.dart 40:61                                                 main
/var/folders/ly/j71v2zl54n74gp72pw876xv800k69x/T/flutter_test_listener.MCatGI/listener.dart 47:18  main..
===== asynchronous gap ===========================
package:test_api                                                                                RemoteListener.start
/var/folders/ly/j71v2zl54n74gp72pw876xv800k69x/T/flutter_test_listener.MCatGI/listener.dart 17:25  serializeSuite
/var/folders/ly/j71v2zl54n74gp72pw876xv800k69x/T/flutter_test_listener.MCatGI/listener.dart 43:36  main
✖ loading /Users/dnfield/src/flutter/flutter/packages/flutter/test/widgets/image_test.dart
Exited (1)
This particular test is not a golden test (even though it's in
image_test.dart).
/cc @Piinks https://github.com/Piinks
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/flutter/flutter/issues/48506?email_source=notifications&email_token=AEBNU3AIEFYAWUCRKX2ZSS3Q45EFDA5CNFSM4KE22A7KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IFDK26A,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEBNU3BL3LWLFJXA232LPD3Q45EFDANCNFSM4KE22A7A
.

--
Kate Lovett |  Software Engineer |<EMAIL_ADDRESS>|  +1<PHONE_NUMBER>

Discovered because my kids unplugged my wifi as I was trying to run a test :D

We should see if we can add a test with an http override that always fails or something

We should see if we can add a test with an http override that always fails or something

There is a test already that should be checking for this: https://github.com/flutter/flutter/blob/82262d8896d58f6d13893d695559a9dc8036cfbd/packages/flutter_goldens/test/flutter_goldens_test.dart#L870
Could this regression have to do with the changes we've been making to our error handling?

That test also fails if you're actually offline because it runs pub.

Talked offline - there's a check at https://github.com/flutter/flutter/blob/51e24a3561ee4f50a2b4ddeb36b1f7ea59d768d2/packages/flutter_goldens/lib/flutter_goldens.dart#L563 for OSError, we should also be catching SocketException there (which is not an OSError).


trustee interface link is broken

trustee interface link is broken. Can you please upload the trustee interface again?

Unfortunately, this is not a maintained charm. It was written a long time ago as an experiment. I may update it in the future if people really want to use it, but there is no timeframe, as I have a pretty big backlog to go through first.


Add draft CONTRIBUTING.md

Right now this is only a draft.
Related issue: #60
TODO/Decisions:

[ ] Details setting up testing using Docker locally (can someone else give me a suggestion for this?)
[ ] Brasil folks: What do you want to say here?
[ ] Decide whether to make the 'sub-official' Telegram official and put it here and/or README.md
[ ] Decide whether to give commit message formatting guidelines?
[ ] Decide if documentation about testing should go somewhere else?
[ ] Details about manual Firebird db setup and setting up an alias using aliases.conf
[ ] Platform specific notes
[ ] Other additions/suggestions?
[ ] Links to/from other docs
[ ] Inter-section links
[ ] Table of contents

Pull Request Test Coverage Report for Build 269792634

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-32.2%) to 45.442%

Totals

Change from base Build 267927205:
-32.2%

Covered Lines:
1296

Relevant Lines:
2852

💛  - Coveralls

Decide whether to make the 'sub-official' Telegram official and put it here and/or README.md

I think some web service may be better where you do not have to give a phone number. I use Telegram for some things but maybe it is a little privacy concern for some people ( I feel this way a little bit )
I will look at gitter.im.  it's not obvious to me if their interface is english-only.
any other suggestions here?

Decide whether to make the 'sub-official' Telegram official and put it here and/or README.md

I use the telegram because the activity of rust group there. We can use the telegram for now and make a gitter account too.

I think some web service may be better where you do not have to give a phone number. I use Telegram for some things but maybe it is a little privacy concern for some people ( I feel this way a little bit )

You can hide your phone number in the telegran.

any other suggestions here?

We can use a mailgroup into google groups, like the firebird-support

* Other additions/suggestions?

We can create the README.md for each crete, referring his own docs. Today we only have a link to the main crate docs

voila https://gitter.im/rsfbclient/rsfbclient (gitter is a bit funny about naming communities vs rooms. maybe it's the same on other sites)
@fernandobatels and @jairinhohw
I hope you do not mind that I send you invitations. If you don't want to use gitter.im at all, that's OK

Continuing on #107


UTF8 marker for exercise and solution

The UTF8 markers in the titles help readability. We should add some for the exercise and their solutions.

I propose 📝 for exercise and 📃 for solution
Firefox:

Chromium:

Alternative solution more than welcome:
https://unicode.org/emoji/charts/full-emoji-list.html

I love your proposal. Thank you!

There was also 🔓 but to me it is not clear enough that the lock is unlocked (on Firefox) ...


Paralel processing with/without iPython parallel

Hi,
I am little bit confused about the contrast between bcbio_nextgen.py -n [NUMBER] bcbioconf.yaml and ipython parallel. Although, I have read your article for Variant calling – benchmarking AWS versus local, I could comprehend well. Sorry.
My aim is to do this process as fast as possible. (Amazon stuff did not go as expected so try to build bcbio in local hpc of university).
My university will allocate ~150 cores to this process. so we are thinking about running 5 x 30 cores at the same time. ( 5 different bcbioruns with different samples in one big cluster).
To understand better I have these following questions;

If I use bcbio_nextgen.py with only -n [NUMBER] option. Will bcbio still execute in parallel fashion.
If I run the same process with all the same options but ipython on, will I gain time ?
What would be the benefit of the ipython? ( forgive my ignorance)
Would you guide me to the way to get best performance ?(with/without ipython)

Thank you for the valuable information,
Best regards,
Tunc.

Tunc;
Running with ipython allows you to parallelize a run over multiple machines. If you run with -n only in local mode it can still parallelize, but over the cores available on a single machine. It sounds like if you're on a compute cluster and have 5 30 core machines you want to use ipython to make use of all 150 cores. The documentation has more details about parallelization:
http://bcbio-nextgen.readthedocs.org/en/latest/contents/parallel.html
Hope this helps.


Fix typos in specification

Checked with hunspell and aspell in Linux command line. This mostly fixes some real typos like "identifer" and "resouce", but also normalizes some acronyms like CGSpace, AGROVOC, and MARLO.
For the language, eventually we could normalize against en_GB or en_US because it's currently a bit of both right now (ie "harmonised" and "organization"). This is easy in Linux:
$ aspell check -d en_GB

thanks @alanorth, I didn’t know the aspell command.
Merging as it is now fine I think


Overwriting LD_LIBRARY_PATH

On musl libc systems where we're setting LD_LIBRARY_PATH when exec'ing the our dynamic loader, we're overwriting the existing value of the environment variable if it's set already. We should prepend our $libdir to the existing value instead in that case.

There are no longer any active references to LD_LIBRARY_PATH in our codebase. Closing.


[3.x] Backport the GLTFDocumentExtension system

This PR backports the GLTFDocumentExtension system, including #66026, #66094, #67428 #68981, #69022, #69925, and part of #52802 to Godot 3.x (still only works in editor builds, not in game templates).
This allows extending the GLTF importer either in a module or in GDScript (probably GDNative too). I have tested in GDScript and also tested backporting the physics PR (not included in this PR).

Thanks!


Alekhine Defense: Exchange Variation, Voronezh Variation

The Voronezh Variation of the Alekhine Defense as it is referred to e.g. in Play the Alekhine Defence (Alexei Kornev)
Starting Out: Alekhine’s Defence (John Cox).

Thanks!


LongPath support

The new long path support for Windows 10 CU had to be backed out again, because it failed on lower windows versions.
https://github.com/fsprojects/Paket/pull/1944
Another option would be to use libraries like https://github.com/peteraritchie/LongPath or https://github.com/alphaleonis/AlphaFS, which replace the System.IO namespace. These have the added advantage that they should also work on all Windows versions, not only Windows 10 CU.

https://github.com/fsprojects/Paket/pull/1944#issuecomment-316843735 could work as well.

Just a quick note to express interest in getting this fixed. I'm having a real hard time using GitHub github actions with a self-hosted runner now because github actions checks out my code into .../_work/long-repo-name/long-repo-name and when paket goes to restore NuGet dependencies I get paths like C:\actions-runner\_work\long-repo-name\long-repo-name\packages\test\runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl\runtime.opensuse.13.2-x64.runtime.native.system.security.cryptography.openssl.4.3.3.nupkg which paket can't handle.

Hi Eric, there's a workaround: use  an external manifest together with the app config switch.
Create a new file paket.exe.manifest with the following content (and place it next to paket.exe)
<application xmlns="urn:schemas-microsoft-com:asm.v3">
  <windowsSettings>
    <longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
  </windowsSettings>
</application>

Add the required switch to Paket.exe.config:
<!-- see https://blogs.msdn.microsoft.com/jeremykuhne/2016/07/30/net-4-6-2-and-long-paths-on-windows-10/ -->
        <AppContextSwitchOverrides value="Switch.System.IO.UseLegacyPathHandling=false;Switch.System.IO.BlockLongPaths=false" />
    </runtime>
</configuration>

Increase the last-modified timestamp of paket.exe or reboot because windows caches the existence or absence of a manifest.
See https://github.com/fsprojects/Paket/pull/1944#issuecomment-611646702 for context.


Aborting the http request does not abort async resolvers

Product
Hot Chocolate
Version
13.9.12
Link to minimal reproduction
https://github.com/jarlef/HC-Bug-CancellationToken
Steps to reproduce

Run server and query the foo field resolver in BananaCakePop
Click "Cancel" immediately
Observe HC has returned ClientAbortError but the resolver is still being executed without raising OperationCanceledException

What is expected?
HC should raise OperationCanceledExceptionon the CancellationToken provided to the async resolvers
What is actually happening?
The client receives an instant confirmation that the request has been aborted but HC server is still processing
Relevant log output
No response
Additional context
Have also added a asp .net minimal api get endpoint which performs the same logic, but it manages to raise the OperationCanceledException and abort the request properly.
Not this also affects older version of HC down to at least 13.5.0 (or even further) and also 14.0.0-rc.1
This is real concern for APIs that are invoking slower upstream apis that the resolvers are not terminated and thereby performing heavy computation even if the client has determined the computation is no longer needed. Essentially clients could requests could create a DDoS situation.

This is not a Hot Chocolate bug but a BCP issue ... @rstaib it seams that the request is not correctly aborted in BCP. I checked with postman and aborting the request in postman leads to the correct cancellation.

We first notice the same issue using react apollo where we call the abort on the query request. The requests turns up as cancelled in chrome but the server still processes the request. Will check this further if we can reproduce this.

So we tested it now intensely and it works like with BCP, react and postman. We do not see any issue. So we cannot reproduce the issue you're having.
Sometimes  this is cause by an invalid request middleware or by http request interceptor that does not pass the original cancellation token on.
I am closing this issue as this seem a user problem. If you however produce a repro that is verifiable you can file a new issue.

@michaelstaib  Turns out my repro was inaccurate. Seems like aborting requests in banana cake pop (or in general) in Brave does not trigger  asp .net to raising the OperationCanceledException. That seems to be a browser issue (i'm going to switch browser from now on)
However. I found out that async resolvers aborts when the request is aborted in chrome. But if the async resolver is using dataloader, then the dataloader is not aborted.  I have updated with a repro with an example using dataloaders. I have attached a http file with the different request (with and without data loaders)

I'm will create a new issue

https://github.com/ChilliCream/graphql-platform/issues/7470


Material Design Feature Discovery

https://material.io/archive/guidelines/growth-communications/feature-discovery.html#
Pieces flutter should probably provide (and demo in the Gallery):

[ ] Tap Target colored overlay with help text and pulsing target area
[ ] Straightforward way to trigger ripple effects on certain elements during guided flows
[ ] Guided Flow helper class of some form to help orchestrate one of these.
[ ] Completion Toast (just a styled active-colored bottom bar?)
[ ] Hint Text widget

Thoughts @Hixie @HansMuller?

I think this is really important feature, mobile apps are getting rather complicated, and it's not like users are going to read a manual.
I'll put demoing the features you've listed at the top of my short stack.

Moving this out from Gallery App milestone. We can ship the gallery without this feature. It's a good feature for a v1.1 or later.

Leafy currently implements this in the native android app; we had to rely on the following features of Android platform to make it happen. Flutter solution might be drastically different:

Find any widget in the widget tree and discover its coordinates on screen on layout complete.
If the widget is currently not visible on screen, we should be able to find the scrollable parent and do programmatic scroll to make it visible.
Listen on scroll events to fire an event on scroll complete; i.e. be able to detect when the widget is now fully visible and the UI has settled.
Overlay a view on top of the screen which prevents the user from interacting with the app. It should consume all gestures. Tapping on this view should advance the guided flow (or end it).
Some way to highlight the aforementioned widget. We originally tried animating the background by triggering ripple but we found out that users didn't understand/discover that. So it would be better to hide irrelevant parts of the app:
Material does this: https://material-design.storage.googleapis.com/publish/material_v_8/material_ext_publish/0B4kv-3uZbS1qeGJFTWZ0M2R6ZEE/useredu_er_featuredisc_design_01.png
We do this:

We achieved it by painting a gray rectangle and using "clear" paint to punch a hole. We then blurred the whole thing using renderscript and overlaid it. It is extremely hacky but it works well.

Finally, a way to lay help text at specific coordinates on the screen. This can be part of the overlay widget itself.

If these capabilities are provided, we can contribute to flutter by writing a user-friendly version of it so that a list of widget tags and help texts is the only thing needed for the guided tour to run.

My understanding is that this is not on leafy's top-asks list at current, so bumping this to M2.

This is a bigish project for someone once we do it, and with so much on our near-term plate, I think we should wait until it's really an issue.

@eseidelGoogle : Are the demos mentioned in the first comment still required, if so I can work on them.

I can potentially work on these too.
But there's one thing I wonder :  What are the rules behind the first video about TapTarget here ?

As it doesn't seems to fit mobile rules. Shouldn't the example be similar to

?

It would be great if Flutter team could provide an official Feature Discovery library with long term support

I think this is deprecated?! It was removed from material specs and only lives in the archive.
I would like to know what was the study and motivation from Google to remove that, but unfortunately, I was not able to find or they don`t share the reasoning.
I think this is the new way to do that:
https://material.io/design/communication/onboarding.html

CC @willlarche

It has not been updated in a few years since it's lower in priority. But we have an internal version we'd like to open source that mirrors the expected existing spec.

This is a crucial feature in making the app more user-friendly. Every now and then I visit this issue hopefully for checking any updates.
@willlarche Please consider contributing this feature to the official Flutter repo or publishing this as a stand alone package like the Google's material design widgets for Android.

@willlarche Is this internal version already Flutter code?
Then publishing it might even be beneficial without any polishing as it can take away a bunch of work from people (like me) trying to implement this.

is feature discovery still part of the MD specs? is not present in the docs anymore, just in the archive.

I've been using the lib @ayalma mentioned now for many weeks.  I love it!  I've also been able to add commands right in the discovery text to "dismiss" or "skip tour" so users can skip all of them if they want.  And then a button under account profile to "take the tour" again.
Here are some code snipets:
  void initState() {
    SchedulerBinding.instance.addPostFrameCallback((Duration duration) {
      if (SessionInfo?.of(context)?.mySkipTour == false) {
        FeatureDiscovery.discoverFeatures(
          context,
          const <String>{
            'search',
            'defaults',
            'add_new',
          },
        );
      }
    });

                              child: DescribedFeatureOverlay(
                                featureId:
                                    'search', // Unique id that identifies this overlay.
                                tapTarget: Icon(
                                  Icons.search,
                                  color: Theme.of(context).accentColor,
                                  size: 44,
                                ),
                                title: Text('Search For Spaces and Devices'),
                                backgroundColor: Theme.of(context).primaryColor,
                                targetColor: Colors.white,
                                textColor: Colors.white,
                                contentLocation: ContentLocation.below,
                                onDismiss: () =>
                                    featureDiscoveryDismiss(context),
                                description: Column(
                                  crossAxisAlignment: CrossAxisAlignment.start,
                                  children: <Widget>[
                                    const Text(
                                        'Tap the search icon to search for specific serial numbers, device names, or space names.'),
                                    Container(
                                      height: 10,
                                    ),
                                    Row(
                                      children: <Widget>[
                                        FlatButton(
                                            child: Text('Dismiss',
                                                style: Theme.of(context)
                                                    .textTheme
                                                    .button
                                                    .copyWith(
                                                        color:
                                                            lightGreyMenuColor)),
                                            onPressed: () => setState(() {
                                                  FeatureDiscovery
                                                      .completeCurrentStep(
                                                          context);
                                                })),
                                        FlatButton(
                                            child: Text('Skip Tour',
                                                style: Theme.of(context)
                                                    .textTheme
                                                    .button
                                                    .copyWith(
                                                        color:
                                                            lightGreyMenuColor)),
                                            onPressed: () => setState(() {
                                                  SessionInfo.of(context)
                                                      .skipTour();
                                                  FeatureDiscovery.dismissAll(
                                                      context);
                                                })),
                                      ],
                                    ),
                                  ],
                                ),
                                child: FlatButton(
                                    onPressed: () => searchDialog(),
                                    child: Icon(
                                      Icons.search,
                                      color: Theme.of(context).accentColor,
                                      size: 44,
                                    )),
                              ),

Feature Discovery Library is long waited!

Just found this library: https://github.com/SimformSolutionsPvtLtd/flutter_showcaseview

This issue is now nearly 6 years old and although if would be an interesting project for the Flutter team, there are still no plans to do it. The Material Design spec only covered this topic in the first version of the spec, it hasn't been included in any of the updates since about 2018.  So with that said, I am going to (finally) close this issue.


Various problems with building Swift packages

When building Swift packages, there are multiple problems:

The formatting from build output could leak out and make a hard-coded message about Clang modules turn green.
There is an error of build.db missing when compiling a Swift package with the --verbose flag, at least with S4TF.
Two module.modulemap files that declare the same Clang module can overwrite each other, even if one is part of the documentation of a Swift package and never actually involved in the build process. This happened with the modulemap currently in the Utilities directory of s4tf/s4tf.
The headers referenced by a modulemap file are absent when you re-load the package. The -Xcc -I/... flags from when you built the package helped SwiftPM locate headers. Now that LLDB is loading the module with no knowledge of those manual includes, it doesn't know where the headers are.
https://github.com/apple/swift/issues/58916 is still an issue and I don't have a workaround for it.

For some of these problems, a workaround may not be possible or feasible. Thus, I might have to make documentation warning the user to copy certain header files to the system include path. Or, I might have to warn against adding duplicate modulemap files to a GitHub repo and anything it recursively depends on.

Solutions:
1.
2.
3.
4.
5. Seems to be resolved on recent Swift toolchains. Check https://github.com/apple/swift/issues/58916 for the latest information.

I am going to declare problem 4 a permanent workaround. Scanning headers could become theoretically impossible if someone uses C macros to generate the include statements.
Problem 3 is not Swift-Colab's fault; it's SwiftPM's fault.

Actually, there is a potential solution for problem 4 - I'm going to try overhauling the build system and how LLDB loads modules. I could store metadata with each Clang module that asks LLDB to look in those search paths before initializing. There's a one-Jupyter session delay between when the Clang module is generated and when it can be used. This means I can ensure every Clang module gets its proper search paths stored.
The question is: how will I resolve those search paths? The new %install-swiftpm-environment magic could cd and change the working directory. I will force the package to build in a pre-determined directory for unrelated reasons, but this makes resolving -I much easier.


Django Template Navbar - subjects and record links not working

On the record view and record data entry pages the nav bar links for record and subjects do not work.
Navbar record link error:
will return to the subject view page but no external records are showing up. there might be an issue with receiving the protocol number from the URL in the react subject view component.
Navbar subjects link error:
this will return to the project view page with a list of subjects for given protocol, however you are not able to select an individual subject.
Django Template Navbar locations:

individual external record form
external record selection form

oh! oops, sorry about that.
None of my subjects had records (I forgot to do those steps in the set-up) so I didn't get a chance to test the navbar for the record and record-data entry view. I'll look into this!

No worries! This Navbar is not a react component, it is HTML from the django template so I see this as a separate issue - but I do think the underlying issue is within the subjectView react component.
Thanks for looking into it!

Record link is now working - there was an issue with the react logic when rendering the detailed subject view page.
The subject list is still an issue - something is going wrong with the URL when selecting a subject. ... probably more react logic issues.

ok fixed :)


Portable package for windows

@kapitainsky
Thanks again for transforming a zombie sw into a real alive program!
Last 1.7.0 changed the windows version, replacing  the zip package by an installer.
Unfortunately, some users cannot install programs and the installation modifies the registry.
Workaround is to install in another machine, zip the folder to transport to another computer, and remove the installation.
Could you please retain the zip package in the distribution?
Thanks for considering!
CT

You can always install for current user only. It does not require admin rights. Copy files and then uninstall.
Or extract files from installer using e.g.
https://constexpr.org/innoextract/
There are so few users who need it that I consider it misleading for others to have exe and zip files released.

On 1 Dec 2019, at 09:19, caspertone2003<EMAIL_ADDRESS>wrote:

@kapitainsky
Thanks again for transforming a zombie sw into a real alive program!
Last 1.7.0 changed the windows version, replacing the zip package by an installer.
Unfortunately, some users cannot install programs and the installation modifies the registry.
Workaround is to install in another machine, zip the folder to transport to another computer, and remove the installation.
Could you please retain the zip package in the distribution?
Thanks for considering!
CT
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Understood.
Good reference to innoextract, perhaps add it to the information page?
CT

Do you still have problem with tasks file not being saved in the same folder as binary in portable mode?

Good reference to innoextract, perhaps add it to the information page?

Will add. good point


Cache not working?

Hi, just installed shiori on a server and trying it out. Looks great!
Unfortunately, I added a few bookmarks and none of them generated any cached versions. In what conditions does this happen? Is there some dependency that I have to install on my server?
Thanks

Hi @tomas, thanks for using this app and reporting this issue.

Is there some dependency that I have to install on my server ?

No, there are no dependency needed.

I added a few bookmarks and none of them generated any cached versions. In what conditions does this happen?

Normally, shiori should fetch the content by default. May I know how you add the bookmark ? Using command line, web interface or using shiori import ? If possible, could you give me the URL where shiori doesn't fetch any content ?
Thank you.

Via the web interface, although I did try updating the link afterwards from the command line.
One of the links I tried was this small HTML page:
https://tinyapps.org/system.html

Hi @tomas, could you check it again ?
The caching process is done by go-readability, and I've push the fix at 9724ab.

It works! However, links in the copied HTML are all wrong (they're somehow prepending the original link's hostname in them):

Fixed in go-readability#558d9b4.

Great, thanks.
Not related, but, would it be possible to have a bookmark added just by entering a URL and hitting the enter key? Having to use the mouse and click the "Done" button affects usability IMHO.

Oh and it looks like go-readability's content lookup doesn't work always. Here's another URL where I get an empty result: https://www.plop.at/en/bootmanager/index.html

Not related, but, would it be possible to have a bookmark added just by entering a URL and hitting the enter key? Having to use the mouse and click the "Done" button affects usability IMHO.

Yeah, it's in the future plan.

Oh and it looks like go-readability's content lookup doesn't work always. Here's another URL where I get an empty result.

Yeah, it's still in development and I haven't finished porting the entire methods from mozilla/readability. There are some parts of code that I find quite hard to port, especially since Go and JS have opposite character (static vs dynamic language), so PR and contributions are always welcome.
BTW, since go-readability is port of mozilla/readability (which powers reader mode in Firefox), that means if Firefox unable to render the readable format for a website, most likely go-readability can't do it as well.

I see. golang isn't my weapon of choice, so unfortunately I cannot help you much with PRs in go-readability. Thanks again!


Require CPIO support

My script entered an infinite loop (during "waiting for installer to finish") because cpio operations failed.

Merged #41


Re-format files in source, source/opt, source/util, source/val and tools

NFC. This just makes sure every file is formatted following the
formatting definition in .clang-format.

Re-based and pushed as d2938e48427cb6e8d5996712c23496d62e3c08d1


Support a default output dir name by convention and auto fill that dir name in CICD template's terraform_root

Stale issue. Will reopen if needed.


Make "public"

I'm creating this issue as a reminder (and maybe to have a checklist) to make this repository "public".

DONE.


Probably don't need mutable control for visitor

Because we have Action::Change to change control nodes, I don't think that we need to give each visitor function a mutable reference to its control statement.

This might simplify some of the visitor code and maybe make the RefCell in context.rs unnecessary


Crash

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x60 pc=0x7d1461]

goroutine 35 [running]:
github.com/davidnewhall/unpackerr/pkg/unpackerr.(*Unpackerr).checkExtractDone(0xc0001759d0)
	/unpackerr/pkg/unpackerr/handlers.go:147 +0x3a1
github.com/davidnewhall/unpackerr/pkg/unpackerr.(*Unpackerr).Run(0xc0001759d0)
	/unpackerr/pkg/unpackerr/start.go:212 +0x1c5
created by github.com/davidnewhall/unpackerr/pkg/unpackerr.Start
	/unpackerr/pkg/unpackerr/start.go:153 +0x3b6

2022/05/21 07:51:05 Unpackerr v0.10.0 Starting! (PID: 1) 2022-05-21 07:51:05.161003964 -0500 CDT m=+0.048427727
2022/05/21 07:51:05 Missing Lidarr URL in one of your configurations, skipped and ignored.
2022/05/21 07:51:05 [ERROR] Checking Radarr Path: redirected to login page while checking URL http://<IP_ADDRESS>:7878: API Key may be incorrect
2022/05/21 07:51:05 [WARN] Sonarr URL fixed: http://<IP_ADDRESS>:8989 -> http://<IP_ADDRESS>:8989/sonarr (continuing)
2022/05/21 07:51:05 ==> GoLift Discord: https://golift.io/discord <==
2022/05/21 07:51:05 ==> Startup Settings <==
2022/05/21 07:51:05  => Sonarr Config: 1 server: http://<IP_ADDRESS>:8989/sonarr, apikey:true, timeout:15s, verify ssl:false, protos:torrent, delete_orig: false, delete_delay: 5m0s, paths:["mnt/user/mount_mergerfs/GSDrive_Mount/completed"]
2022/05/21 07:51:05  => Radarr Config: 1 server: http://<IP_ADDRESS>:7878, apikey:true, timeout:15s, verify ssl:false, protos:torrent, delete_orig: false, delete_delay: 5m0s, paths:["mnt/user/mount_mergerfs/GSDrive_Mount/completed"]
2022/05/21 07:51:05  => Lidarr Config: 0 servers
2022/05/21 07:51:05  => Readarr Config: 0 servers
2022/05/21 07:51:05  => Folder Config: 0 paths, event buffer: 20000
2022/05/21 07:51:05  => Using env variables only. Config file not found.
2022/05/21 07:51:05  => Parallel: 1
2022/05/21 07:51:05  => RAR Passwords: 0
2022/05/21 07:51:05  => Interval: 2m
2022/05/21 07:51:05  => Start Delay: 1m
2022/05/21 07:51:05  => Retry Delay: 5m, max: 3
2022/05/21 07:51:05  => Debug / Quiet: false / false
2022/05/21 07:51:05  => Directory & File Modes: 755 & 644
2022/05/21 07:51:05  => Log File: /downloads/unpackerr.log (10 @ 10Mb)
2022/05/21 07:51:05  => Webhook Configs: 0 URLs
2022/05/21 07:51:05  => Command Hook Configs: 0 cmds
2022/05/21 07:51:05 [ERROR] Radarr (http://<IP_ADDRESS>:7878): api.Get(queue): failed:  (status: 302 Found): invalid status code, <200||>299:
2022/05/21 07:51:05 [Sonarr] Updated (http://<IP_ADDRESS>:8989/sonarr): 777 Items Queued

If you're getting a crash like this, try the docker tag golift/unpackerr:unstable and let me know if it fixes it for you.

Looks good to me. No crashing anymore. Ran it overnight with the same issue beforehand.

Thanks I'm going to get a new release - with this fix - out soon.


Look into running github actions (pre-commit, tests)

Look into running github actions (pre-commit, unit tests, integration tests).  First pass, we definitely want pre-commit running.  Then I'd say unit tests and integration tests, though the latter might be a bit tougher.

pre-commit hooks done with PR #90

Next step is unit tests.

Unit tests are mostly done.  We now really need good integration tests.  @marcmengel has done most of the tests we need already, but we should expand on that.
Related to #243


Allow using cargo-binstall / cargo-quickinstall to download install_crate dependencies

Feature Description
Building crates from scratch can be slow—luckily cargo-quickinstall and cargo-binstall provide a solution! Popular crates (in the case of quickinstall) or crates that opt-in to the right format (in the case of binstall) can be downloaded as prebuilt binaries! I'd love to be able to use install_crate in my Makefile.toml while leveraging one of those tools; unfortunately, neither of them have a --force flag, so I can't get this to work.
Describe The Solution You'd Like
Add a no_force flag or similar to install_crate to allow omitting the --force flag so that install_command = "binstall" will work.
Code Sample
[tasks.mdbook.install_crate]
crate_name = "mdbook"
binary = "mdbook"
test_arg = ["serve", "--help"]
install_command = "quickinstall"

Today, the output is:
Error: "unexpected positional arguments: --force mdbook"

good idea. I'll check it out

@dbanty can you checkout the dev branch 0.35.16 for this?
basically add also force=false to the install_crate block.
still need to write few tests, but it should be ok now.
would love your feedback

Hmm, I'm still seeing the same error 🤔 I could have done something wrong though. I installed the new version like cargo install --git https://github.com/sagiegurari/cargo-make --branch=0.35.16 (I've never used that syntax before, so maybe that's wrong?). It then told me:
Replaced package `cargo-make v0.35.15` with `cargo-make v0.35.15 (https://github.com/sagiegurari/cargo-make?branch=0.35.16#7b52723d)` (executables `cargo-make`, `makers`)

I then uninstalled mdbook with cargo uninstall mdbook and added a couple of lines to my install_crate section so that it now looks like this:
[tasks.mdbook.install_crate]
crate_name = "mdbook"
binary = "mdbook"
test_arg = "--help"
install_command = "quickinstall"
force = false

When running a makers serve-book command that requires that dependency, I still get this output:
[cargo-make] INFO - makers 0.35.15
[cargo-make] INFO - Project: knope
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: serve-book
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: legacy-migration
[cargo-make] INFO - Execute Command: "cargo" "quickinstall" "--force" "mdbook"
Error: "unexpected positional arguments: --force mdbook"
[cargo-make] ERROR - Error while executing command, exit code: 1
[cargo-make] WARN - Build Failed.

thanks and sorry about that. i'll check it out tomorrow. didn't have time to setup tests...

No problem, thanks for working on this!

@dbanty it should work well now with your usecase.

@dbanty this is now officially released. if there are any issues with it, feel free to open an issue and I'll handle.
thanks for the idea


[FirebaseAuth] Error 17006, The given sign-in provider is disabled

Description
Hello!
Today, we encountered an issue with Firebase Authentication. When I attempt to verify a phone number using the PhoneAuthProvider.provider().verifyPhoneNumber method, I consistently receive the following error: "The given sign-in provider is disabled for this Firebase project. Enable it in the Firebase console, under the sign-in method tab of the Auth section."
I want to confirm that the phone sign-in provider is indeed enabled in the Firebase console, and SMS verification is activated for all regions.
Everything was functioning perfectly until today, and we haven't made any changes. However, the error started occurring unexpectedly.
Additionally, we discovered that the verification works correctly for US phone numbers, but it fails for Hungary numbers, producing the error mentioned above. We also attempted to add Hungary to the SMS verification regions list directly, but it didn’t resolve the issue.
We would appreciate your assistance in resolving this matter. It seems like it could be a bug.
Thank you!
Reproducing the issue
Integrate the iOS SDK, try to login with a HU number.
Firebase SDK Version
10.24.0
Xcode Version
15.2 (15C500b)
Installation Method
CocoaPods
Firebase Product(s)
Authentication
Targeted Platforms
iOS
Relevant Log Output
Error Domain=FIRAuthErrorDomain Code=17006 "The given sign-in provider is disabled for this Firebase project. Enable it in the Firebase console, under the sign-in method tab of the Auth section." UserInfo={NSLocalizedDescription=The given sign-in provider is disabled for this Firebase project. Enable it in the Firebase console, under the sign-in method tab of the Auth section., FIRAuthErrorUserInfoNameKey=ERROR_OPERATION_NOT_ALLOWED}

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

Replace this line with the contents of your Package.resolved.

 I had the same problem but using Peru phone numbers. Fortunately I was able to resolve (workaround) it by going to the Authentication settings in the firebase console, selecting SMS region policy, switching from Deny to Allow and then adding my supported countries explicitly.
Not sure what changed in the past week for Firebase that broke this. Also they should fix the error code because 17006 is not accurate based on the error message. Also a test phone number should always work regardless of SMS region policy.

@cvance Not really, it was likely an issue on their backend. I've seen other posts mentioning that certain phone numbers didn't work, mainly due to problems with the third-party services Firebase uses for sending SMS messages.

@rizafran this error still occurs I have some test phone numbers that don't work, obviously I don't want to make them public, is there a way to send them your way for you to test?


Simplify Amplify CDK connection

At the moment we need to run complex scripts to parse a config file for Amplify. It would be cool to forward the CDK references to amplify on runtime.
https://github.com/aws/aws-prototyping-sdk/discussions/80

Currently blocked because of https://github.com/aws/aws-prototyping-sdk/issues/84


[IGNORE] BIDS Validator XS

Draft PR to track the diff for a smaller BIDS validator.

NOTES from meeting:

Example data for testing: https://github.com/bids-standard/bids-examples/
Repository of path-accessible schema versions, only https://github.com/bids-standard/bids-schema , older versions will need to be added, which might break due to logic updates — potentially old schemas can be updated wrt. logic but not content. This might shape up to be a lot of work when updating schema logic.

@tsalo @yarikoptic path validation should be done pretty reliably now, and a slight variation of the code is already used “in production” as part of DANDI. I think this is ready for a review.
Only thing missing are integration tests (commented out right now), since I'm unsure how you would like to handle the reference data download. DANDI uses a fixture, though maybe a static archive could also do, or even an included archive. The issue with pulling from git is that tests might break due to asynchronous test data and validator updates. Though maybe that's desired to make enough noise to ensure the code always stays up to date....

@effigies @tsalo updated it to work after the latest update following the hackathon, also has good test coverage. Not fully featured as of right now (only checking filenames, still, and not using the error codes), but I think merging the current version already offers good value, and I can add further features incrementally, possibly in sync with what we consider most definitive-ish.

Does not seem to validate sidecar files at the root of the dataset:
2022-05-19 14:25:52,937 [ WARNING] The /home/rwblair/projects/datasets/ds000003/task-rhymejudgment_bold.json file was not matched by any regex schema entry.

@effigies loading variable field strings from the schema now. You said we could merge before the inheritance thing? It's in the notes, but might be best for a separate PR.


copy2clipboard linux does not work

You have to install "python3-pyperclip" in ubuntu to make the copy function work from pyperclip3.


Error message no Build Task found, when there is a build task defined in the task.json

I've added an attribute to the taskDefinition of the npm task provider:
    "taskDefinitions": [
      {
        "type": "npm",
        "required": ["script"],
        "properties": {
          "script": {
            "type": "string",
            "description": "The npm script to customize"
          },
          "path": {
            "type": "string",
            "description": "The path to the folder of the package.json file that provides the script. Can be ommitted."
          }
        }
      }
    ]

Notice that the property path is not defined as a required property.
This is the code to create the task definition:
	let kind: NpmTaskDefinition = {
		type: 'npm',
		script: script,
		path: rootPath
	};

In our smoke test repository we have a setup with an existing task.json file with this contents, that defines a build task:
{
    "version": "2.0.0",
    "tasks": [
        {
            "type": "npm",
            "script": "eslint",
            "problemMatcher": {
                "base": "$eslint-stylish",
                "fileLocation": "absolute"
            },
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]
}

Running Tasks>Run Build Task... results in:

The error message should be more specific and tell that the build task was found but that the definition has issues. However, since the property  path is not required property, I would have expected that the task the existing build task still works.
So there appear to be two issues:

the non-required property in the task definition is not properly handled
the error message is misleading.

Dups https://github.com/Microsoft/vscode/issues/31970


Add item counter test

In this pull request, I implemented the following:

Tested for items counter

Status: ✔ Project Approved ✔
Hi @SelmaNdapanda,
Everything is looking great! Please, merge this branch :shipit:
Congratulations! 🎉


Mocks for "Address Correction" page

Old UX

Design mocks:
https://www.figma.com/design/RmhVJCLgAKWGENJLWWAtNS/CiviForm%3A-Components%2C-Styles?node-id=3705-15213&m=dev

Here are some screenshots of the case where there are more than 0 suggested addresses.
Old (Legacy) UX

WIP in Thymeleaf


[WIP] Update text classification template labels in DatasetInfo post_init

Hi @yjernite and @lhoestq, here's a first stab at the suggestion discussed in #2389 to update the labels of the TextClassification template in the DatasetInfo.__post_init__.
One problem I've spotted is that my current implementation introduces state into the __post_init__:

When we call load_dataset, DatasetInfo.features are the "raw" features without any casting so we can access the column names by the label_column specified in TextClassification
When we call Dataset.prepare_for_task we run into a problem because the DatasetInfo.features are first cast into the new schema which triggers a KeyError when we update the infos here.

Here's an explicit example of what I mean with the stack trace appended below:
from datasets import load_dataset

# this works
ds = load_dataset("emotion")
# we can verify the task template is correctly set
ds["train"].info.task_templates # returns [TextClassification(labels=('sadness', 'joy', 'love', 'anger', 'fear', 'surprise'), text_column='text', label_column='label')]
# but this fails because the _post_init__ is looking for the original column names
ds.prepare_for_task("text-classification")

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-4-54a43019b319> in <module>
----> 1 ds.prepare_for_task("text-classification")

~/git/datasets/src/datasets/dataset_dict.py in prepare_for_task(self, task)
    807         """
    808         self._check_values_type()
--> 809         return DatasetDict({k: dataset.prepare_for_task(task=task) for k, dataset in self.items()})

~/git/datasets/src/datasets/dataset_dict.py in <dictcomp>(.0)
    807         """
    808         self._check_values_type()
--> 809         return DatasetDict({k: dataset.prepare_for_task(task=task) for k, dataset in self.items()})

~/git/datasets/src/datasets/arrow_dataset.py in prepare_for_task(self, task)
   1421         dataset = self.remove_columns(columns_to_drop)
   1422         dataset = dataset.rename_columns(column_mapping)
-> 1423         dataset = dataset.cast(features=template.features)
   1424         return dataset
   1425

~/git/datasets/src/datasets/arrow_dataset.py in cast(self, features, batch_size, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, num_proc)
    970         format = self.format
    971         dataset = self.with_format("arrow")
--> 972         dataset = dataset.map(
    973             lambda t: t.cast(schema),
    974             batched=True,

~/git/datasets/src/datasets/arrow_dataset.py in map(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint)
   1583
   1584         if num_proc is None or num_proc == 1:
-> 1585             return self._map_single(
   1586                 function=function,
   1587                 with_indices=with_indices,

~/git/datasets/src/datasets/arrow_dataset.py in wrapper(*args, **kwargs)
    173         }
    174         # apply actual function
--> 175         out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    176         datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    177         # re-apply format to the output

~/git/datasets/src/datasets/fingerprint.py in wrapper(*args, **kwargs)
    338             # Call actual function
    339
--> 340             out = func(self, *args, **kwargs)
    341
    342             # Update fingerprint of in-place transforms + update in-place history of transforms

~/git/datasets/src/datasets/arrow_dataset.py in _map_single(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset)
   1959         if update_data:
   1960             # Create new Dataset from buffer or file
-> 1961             info = self.info.copy()
   1962             info.features = writer._features
   1963             if buf_writer is None:

~/git/datasets/src/datasets/info.py in copy(self)
    274
    275     def copy(self) -> "DatasetInfo":
--> 276         return self.__class__(**{k: copy.deepcopy(v) for k, v in self.__dict__.items()})
    277
    278

~/git/datasets/src/datasets/info.py in __init__(self, description, citation, homepage, license, features, post_processed, supervised_keys, task_templates, builder_name, config_name, version, splits, download_checksums, download_size, post_processing_size, dataset_size, size_in_bytes)

~/git/datasets/src/datasets/info.py in __post_init__(self)
    174                     # The reason is that Dataset.prepare_for_task calls Dataset.cast which converts the
    175                     # DatasetInfo.features to the new schema and thus template.label_column is no longer a valid key
--> 176                     object.__setattr__(template, "labels", tuple(self.features[template.label_column].names))
    177                     template.label_schema["labels"] = ClassLabel(names=template.labels)
    178                     self.task_templates[idx] = template

KeyError: 'label'

What do you think? I did this a bit quickly, so maybe I'm overlooking something obvious :) One thing would be to only update the labels of the task template on load, but this seems a bit hacky IMO

If I'm not mistaken, one way to fix this would be to drop the task templates when copying the info by inserting dataset.info.task_templates = None before the Dataset.cast call in Dataset.prepare_for_task. Moreover, we should do this change independently of the KeyError being raised because currently the following is possible:
dset = load_dataset("some_dataset") # let's say 'some_dataset' supports text classification and question answering
dset_tc = dset.prepare_for_task("text-classification")
dset_tc.preprare_for_task("question-answering") # this should raise an error because the schema is no longer valid for this task; currently this fails on 'rename_columns'

I see 2 options:

to drop the task templates after the first Dataset.prepare_for_task call
to save only the tasks compatible with the new schema after Dataset.prepare_for_task` (but then we have to update the column names of the compatible tasks to make sure the column mapping is still valid)

If I'm not mistaken, one way to fix this would be to drop the task templates when copying the info by inserting dataset.info.task_templates = None before the Dataset.cast call in Dataset.prepare_for_task. Moreover, we should do this change independently of the KeyError being raised because currently the following is possible:
dset = load_dataset("some_dataset") # let's say 'some_dataset' supports text classification and question answering
dset_tc = dset.prepare_for_task("text-classification")
dset_tc.preprare_for_task("question-answering") # this should raise an error because the schema is no longer valid for this task; currently this fails on 'rename_columns'

I see 2 options:

to drop the task templates after the first Dataset.prepare_for_task call
to save only the tasks compatible with the new schema after Dataset.prepare_for_task` (but then we have to update the column names of the compatible tasks to make sure the column mapping is still valid)

thanks for the great idea @mariosasko and for spotting the problem with sequential task preparation! i am in favour of your option (1) since it is simple and saves us from having to keep track of the column mappings across multiple steps.
i've implemented the change and refactored the tests to account for the new approach (including a new test that the templates are flushed after we call prepare_for_task). perhaps the slightly inelegant aspect here is that if we want to allow the user to set labels in the TextClassification template, then we have two places (DatasetInfo.__post_init__ and TextClassification.__post_init__) where we need to update label_schema.
on the other hand, dropping labels from the TextClassification signature would have the nice effect that users only have to think about column names when defining their tasks.
in any case, i think it would be a good idea to merge #2376 soon as the current PR is touching a lot of the same places in the codebase 😄

cc @SBrandeis who might also be interested in this feature :)

Tests are failing only because the emotion dataset card doesn't pass our dataset card validator (tags are missing), you can ignore this since it's unrelated to this PR.

@lhoestq @SBrandeis i've fixed the tests and think this is now in a good state for another review :)

Maybe @SBrandeis you can also take a look to make sure you're fine with it ?


time returned in queries using math is corrupt

Influxdb version: 1.5.2
Three queries:

SELECT MIN(field) FROM table WHERE (host = 'localhost' AND instance = 'fred') AND time > now() - 3d GROUP BY host
2)
SELECT MAX(field) FROM table WHERE (host = 'localhost' AND instance = 'fred') AND time > now() - 3d GROUP BY host
3)
SELECT MAX(field) - MIN(field) FROM table WHERE (host = 'localhost' AND instance = 'fred') AND time > now() - 3d GROUP BY host

Time reported by query 1:<PHONE_NUMBER>000000000
Time reported by query 2:<PHONE_NUMBER>000000000
Time reported by query 3:<PHONE_NUMBER>863897245
I cannot comprehend where the time for query 3 comes from. It is not "now".
For reference:
4)
SELECT FIRST(field) FROM table WHERE (host = localhost' AND instance = 'fred') AND time > now() - 3d GROUP BY host

produces this time stamp:<PHONE_NUMBER>000000000
All of the values for "time" in "table" are modulo 60.
Ok, so there was some "lag" time in building this issue but this is reproducible - doing this:
SELECT MAX(field) - MIN(field) ...

always produces a time that is before the first entry and is "random".

@yellowpattern I believe the timestamp is the first interval within the time range specified in the WHERE clause. In the example given above in the issue,<PHONE_NUMBER>863897245 is some time Friday, which I'm guessing was 3d before the query was run (time > now() - 3d).

Yes, it is "some time Friday" but the "some time" part is a rubbish value.
Here's a good example of the problem:

select max(rx) - min(rx) from interface where host = 'localhost' and time > now() - 1d;
name: interface
time                max_min

1526347397942320883<PHONE_NUMBER>5

select last(rx) from interface where host = localhost' and time > now() - 1d;
name: interface
time                last

1526433791549252531 0

select first(rx) from interface where host = 'localhost' and time > now() - 1d;
name: interface
time                first

1526347401549220126<PHONE_NUMBER>6
time(max-min) - time(first)
1526347397942320883-1526347401549220126
-3606899243
This means that the time in the "max() - min()" line is before that for "first()". The above three queries where cut-pasted at the same time.
The time for max() - min() should be the time of the max() value.

Converting those timestamps to human times...

query
time

max(rx) - min(rx)
GMT: Tuesday, May 15, 2018 1:23:17.942 AM

first(rx)
GMT: Tuesday, May 15, 2018 1:23:21.549 AM

Looks like those queries were run in the influx CLI about 3.5 seconds apart. The timestamp returned by the max(rx) - min(rx) query appears to be the first time in the range of time > now() - 1d. Try replacing the relative now() - 1d with the same fixed time for both queries.

select max(rx) - min(rx) from interface where host = 'localhost' and time > '2018-05-15 00:00:00'; select last(rx) from interface where host = 'localhost' and time > '2018-05-15 00:00:00'; select first(rx) from interface where host = 'localhost' and time > '2018-05-15 00:00:00'
name: interface
time                max_min

1526342400000000001<PHONE_NUMBER>6
name: interface
time                last

1526436361549414326 0
name: interface
time                first

1526342401549205023<PHONE_NUMBER>9
max_min - first:
1526342400000000001-1526342401549205023
-1549205022
Try this for yourself.

The timestamp returned by max(rx) - min(rx) is 2018-05-15 00:00:00, which is the start of the query's time range. I think that is the correct behavior.  /cc @jsternberg ?

@yellowpattern  on a side note (and you may already know this), running the following command in the influx CLI will format the result timestamps in human readable form...
> precision rfc3339

Right, well if you want to argue that the behaviour of influxdb is correct then let me tell you that its behaviour is utterly useless in this regard.
This is just a simple example - the original example is from a table where all of the 'time" values in collectd are multiples of 60 seconds. The value returned by max-min is completely f*****d and utterly useless as it does not represent any actual value in the respective table.

@yellowpattern The query is asking for a difference of a field value between two points in time and the result's timestamp is for the first of those two points in time...not the timestamp of one of the actual values between those two points in time. I believe that's the same behavior in the original example as well. I've CC'ed one of the devs that knows that part of the code well.
Can you describe the problem you're trying to solve? Maybe there's another way to get the data you need.

The current behavior for influxql is correct and will not be changed so I'm going to close this issue. Here is the relevant documentation describing the behavior: http://docs.influxdata.com/influxdb/v1.5/query_language/functions/#selector-functions
The version 2 of the query language currently codenamed ifql is still in production and we can change things about how it works. This is one of the things we are changing. ifql will default to using the end time instead of the start time for the timestamp in these situations and there are options to customize the timestamp. If you have any ideas or opinions about how timestamps should work that differ from those ideas, you can open an issue in the ifql repository.
If you do decide to open an issue there, please describe exactly what you want, the output you would expect, and why that output is useful for you so we can ensure that ifql meets the needs of as large an audience as possible. If you need help with how to use influxdb, you can visit our community forum.

For design work for the future of ifql, please consider https://github.com/influxdata/influxdb/issues/9859


ImmediatelyFast

CurseForge link
https://www.curseforge.com/minecraft/mc-mods/immediatelyfast
CurseForge Mod Distribution
Allowed
Modrinth link
https://modrinth.com/mod/immediatelyfast
Source/other link
https://github.com/RaphiMC/ImmediatelyFast
Mod file size
55.98 KB
License
(any other license)
What it does

The following parts of the immediate mode rendering code are optimized:
Entities
Block entities
Particles
Text rendering (partially)
GUIs (partially)

Why should it be in the modpack

Generally FPS should be 2x higher on busy servers and might also be up to 4x higher in cases where there are many entities.

Why shouldn't it be in the modpack
Not on CF, untested
Additional details
No response

Requested a CF upload in https://github.com/RaphiMC/ImmediatelyFast/issues/3

HyperSoop tested it and got inconclusive results - we may not see much benefit with this because Iris already has these optimizations.

seems like it make the performance worse

It's now on CurseForge: https://www.curseforge.com/minecraft/mc-mods/immediatelyfast

New feature coming to immediately-fast said to improve map rendering performance by 5x. (30fps -> 160fps) with a lot of maps.
https://discord.com/channels/507304429255393322/523251999899385875/1056949757924352000

Building a map atlas texture containing 1024 maps each to increase performance of map rendering. This improves performance by over 5 times on servers with billboards or many maps in item frames. (Improved FPS in this scene from 30 FPS to 160 FPS) This feature will be included in the next ImmediatelyFast release.
The image shows the test scene on the left (contains 3 large map billboards) and on the left the allocated textures which are used for rendering. In the after image there are only 2 big map atlas textures instead of the over 500 single map texture vanilla minecraft uses.

A new feature I also added to ImmediatelyFast: HUD/GUI performance improvements. This change is essentially batching various draw calls in the HUD/GUIs and thus increases performance on my system by around 40-50%. In contrast to Exordium is my implementation not too invasive to minecraft code and should pretty much work with most if not all mods. At the moment the following HUD elements are optimized: Chat, Scoreboard, Bossbars, DebugHud, SubtitlesHud, Hotbar, Health/Food bar, PlayerlistHud, status effect overlay.

A new feature I also added to ImmediatelyFast: HUD/GUI performance improvements. This change is essentially batching various draw calls in the HUD/GUIs and thus increases performance on my system by around 40-50%. In contrast to Exordium is my implementation not too invasive to minecraft code and should pretty much work with most if not all mods. At the moment the following HUD elements are optimized: Chat, Scoreboard, Bossbars, DebugHud, SubtitlesHud, Hotbar, Health/Food bar, PlayerlistHud, status effect overlay.

Are the different optimisations toggleable ?
Because it would help for testing specific optimisations

Making them toggleable doesn't make too much sense as they are supposed to all work together to increase FPS. Only having one of the list above won't make a huge (maybe even noticeable) difference. With HUD optimizations its the all the small improvements which add up quickly.
There is also not much point in testing specific optimizations, as the aren't user toggleable anyway. Testing the mod as a whole is the better approach because thats the experience a player would get as well

Making them toggleable doesn't make too much sense as they are supposed to all work together to increase FPS. Only having one of the list above won't make a huge (maybe even noticeable) difference. With HUD optimizations its the all the small improvements which add up quickly.
There is also not much point in testing specific optimizations, as the aren't user toggleable anyway. Testing the mod as a whole is the better approach because thats the experience a player would get as well

The thing is that before the addition of map and HUD improvements i tested the mod and it appeared to do worse, if i was able to toggle optimisation i could precisely determine what was problematic

Given that the modpack contains Iris (Which as stated in the description of my mod) replaces Entity rendering with their own batching implementation. This causes ImmediatelyFast to simply not optimize Entity rendering (and not make it worse because that code won't even run because Iris replaced it as a whole). Disabling only parts of the overall optimizations (Under which all of those areas fall:
Entities
Block entities
Particles
Text rendering
GUIs
Immediate mode rendering of other mods)
is not really feasable as it would make mod compatibility worse (Due to me having to change a lot of vanilla code to force it to run the unoptimized code path) and because I simply can't make settings for optimization of immediate mode rendering of other mods.
The Map and HUD optimizations can be argued with, but I don't see a real reason to make them toggleable, as those are pretty stable and don't have any drawbacks by being enabled. If you still think that adding configuration is necessary I can add it ofc. It just won't do all that much

Given that the modpack contains Iris (Which as stated in the description of my mod) replaces Entity rendering with their own batching implementation. This causes ImmediatelyFast to simply not optimize Entity rendering (and not make it worse because that code won't even run because Iris replaced it as a whole). Disabling only parts of the overall optimizations (Under which all of those areas fall: Entities Block entities Particles Text rendering GUIs Immediate mode rendering of other mods) is not really feasable as it would make mod compatibility worse (Due to me having to change a lot of vanilla code to force it to run the unoptimized code path) and because I simply can't make settings for optimization of immediate mode rendering of other mods.
The Map and HUD optimizations can be argued with, but I don't see a real reason to make them toggleable, as those are pretty stable and don't have any drawbacks by being enabled. If you still think that adding configuration is necessary I can add it ofc. It just won't do all that much

well maybe it shoudn't make it worse but from what i had tried it did make it worse, we have a post dedicated to testing your mod and other people have found the same thing
you should come in our discord to see the results by yourself
https://discord.com/channels/859124104644788234/1026028924813181018/1040688986135474308

If you are going to retry keep in mind that the HUD changes are not yet published on Curse/Modrinth. You have to build it yourself from source (I will publish it today). All of the tests were with Iris installed (Which as I said defeats most of the purpose of the mod. I originally intended it to just be an Iris replacement, as Iris slows down other parts of the game by implementing the whole code for shaders, which it wasn't worth for me just for Entity batching). The test results on the discord channel are really inconclusive because of that. Some reported a 1-3 FPS increase while other report a 3 FPS drop. Thats all well in the margin of error between two runs. The HUD optimizations should make a noticeable difference (on the Hypixel lobby for example). If possible don't use the F3 screen for meassuring performance, thats just not that good because of how much render time the F3 screen itself takes. External tools don't have that overhead and thus show more accurate results

and so i just retested you mod in version 1.0.7 and 1.1.0 in FO
from my tests on hypixel skyblock lobby 1 with 80 players grouped in the hub
max / avg / min
1.1.0 :
48 / 43 / 40
1.0.7 :
45 / 42 /38
whitout :
52 / 46 / 41
what i think is that the map optimisation is clearly effective
but that the entity optimisation has some problem oir smth which lead to result be less good than whitout your mod
that's why i'm asking for a toggle so we can benefit from your map optimisation but disable the entity ones
or at least better diagnostic what's making performance great or bad

Can you give 1.1.1 a try. You can download a testing version on my discord: https://discord.gg/dCzT9XHEWu in #if-general
I still can't really disable entity optimizations, as this is a side effect of my "Universal batching" which allows me to even improve stuff like Maps and HUD. Without that both can not work as they would not use my optimized rendering code path. You could make reports with the spark mod and send me them on my discord or here so I can look whats changing between the two runs. If you do that make sure to test on a multipayer server with a static/consistent scene. Singleplayer might cause too much variance because of the internal server. After starting let the game run a a minute or two to "warm up" before taking the meassurement using spark

1.1.1 is now released and has the HUD rendering optimizations.  Read the update mod description for more details

1.1.1 is now released and has the HUD rendering optimizations.  Read the update mod description for more details

Compensating the performance loss is OK but if actually could be entirely removed and compensated it could be even better
(i'll test and give you feedback and spark as you want tho)

As a said already multiple times: I can't simply remove the code thats responsible for that small performance loss when in combination with Iris. This code is the foundation on which the other optimizations build on. I could reproduce the performance loss on my system as well (2 FPS less for me when I install it together with Iris), so I don't need any spark reports anymore. But I feel like this is a very small issue considering its so small that it can be within the margin of error. Other times reports with Iris show +3 FPS for example. Thats also in margin of error and nothing special

My comparison data using Minecraft Benchmark v3 world:
https://docs.google.com/spreadsheets/d/1ol6HM8a0bIVUmzpBa9VENLQQZ-L3u5Yo_jDsFmL9YNY/edit?usp=sharing

Please retest with IF 1.1.7.
I have added some really nice optimizations in the latest build