[Question] How to add a  tag to ?

We are working on adding Google Tag Manager to a JSS app.
There are two parts in it: specific

I have been suggested to modify /public/index.html for that, which works, but like that it has to be hardcoded. That would work as a workaround, but could you please advise whether there is a way to make it work while also passing the Google Tag Manager container id from Sitecore?

So the way I ended up approaching it is getting rendering's datasource from state.sitecore.route.placeholder["name"].filter(obj => obj.componentName === "RenderingName").fields inserver.jsand editing html via assertReplace
If someone has better suggestions, would be great to hear it - closing the issue for now


Is there support for like TABLE Tool?

I was wondering is there support for a table tool that can be used with the AI
A example of what I'm thinking in PHP.
$TableTool = new AI_Table_Tool();
$TableTool->table(new AI_TT_TabeName("MyPeople", "List of People"));
$TableTool->field(new AI_TT_PrimaryColumn("mp_id", "Primary ID for People"));
$TableTool->field(new AI_TT_IndexedColumn("mp_country_id", "Country ID for this Person"));
$TableTool->field(new AI_TT_SimpleLeftJoin("MyCountryList", "mc_country_id", "mp_country_id"), "Country List");
$TableTool->field(new AI_TT_Field("mp_first_name", "First Name"));
$TableTool->field(new AI_TT_Field("mp_last_name", "Last Name"));
$TableTool->field(new AI_TT_Field("mp_email_address", "Address"));

$chat = new OpenAIChat();
// hostname|username|password|database|new_connection
$chat->addConnection("hostname", "username", "password", "database", true);
// Connection example #1 (store $TableTool against the connection for HOSTNAME and Database)
$chat->addSQLTool("hostname", "database", $TableTool);
// Connection example #2 (store $TableTool to $pdo_link so the AI can access the existing DB and generate SQL)
// https://www.php.net/manual/en/book.pdo.php
$chat->addPDOTool($pdo_link, $TableTool);

$result = $chat->generateText('How many people are using @gmail.com?');
$result = trim($result);

The $result from generateText (SUCCESS) Ideally I think every result should be returned using  PrimaryKey
/* I found 4 people with @gmail.com */
mp_id IN (3, 4, 5, 6)

The $result from generateText (FAIL)
/* There are no people on @gmail.com */
mp_id IN (0)

(OR) The $result from generateText
/* Here is some SQL to find people with @gmail */
mp_email LIKE<EMAIL_ADDRESS>
Another example
$result = $chat->generateText("Give me list of people from Australia");

(SUCCESS) $result
/* Here is everyone from Australia at countryId #2 */
mc_country_id IN (2)

I wasn't sure how to explain, but I hope this describes what I was hoping for.
Could someone possibly build this into the next iteration? or show me an existing example? :)

Hi @killrawr , there isn't such a feature in LLPhant.
IMHO, adding methods like $chat->addConnection or $chat->addSQLTool could mess up the ChatInterface.
I think that one way of doing this would be creating some standard tools to add to the chat via the existing `$chat->addTool' method.

As of now, the tools called are only being executed with no tool response added to the message array. Meaning that LLM can't process the tool call return value.
For that reason, I extended the functionality myself. What I did was to create a LinkedList with Nodes where each node holds Metadata and the Message objects, both of these objects don't care what you attach to them so it was easy to add more properties to Message so it's included in requests. The message class was also extended and added new static methods to create messages straight from the CreateResponse object, this was helpful.
Whenever I received a Message with a ChatRole::Assistant message it was added to the LinkedList as the last message. It's very important to add the Assistant message to the history, and even more important is to have the original value of tool_calls assigned to the message object. Otherwise, it will fail with an API error telling you that the next message with ChatRole::Tool must precede the ChatRole::Assistant message with the tool_calls property.
Then I started to process each tool call (I initially made the mistake of assuming there is one tool call per message). The important bit is to have the tool_id field from each tool call that you can find in the tool_calls array. You will have to use it to answer properly with those tools
Having the function names and arguments in JSON that it wanted to execute was great, but they didn't make it easy to retrieve added tools from the OpenAIChat class, those properties are private. And no method was exposed to retrieve the FunctionInfo so I had to write a separate class to mirror those properties and expose my methods to other parts of my application.
I had to write this class since all the tasks belonged to OpenAIChat. Here is what I came up with
class ToolManager
{
    /** @var FunctionInfo[] */
    protected array $tools = [];

    private OpenAIChat $chat;

    /**
     * @param  FunctionInfo[]  $tools
     */
    public function __construct(array $tools = [])
    {
        foreach ($tools as $tool) {
            $this->addTool($tool);
        }
    }

    public function addTool(FunctionInfo $functionInfo): void
    {
        $this->tools[$functionInfo->name] = $functionInfo;

        // Make the underlying Chat class aware of the tools we have added
        //  if chat is not given, this class serves only as storage
        if (isset($this->chat)) {
            $this->chat->addTool($functionInfo);
        }
    }

    public function getToolByName(string $name): ?FunctionInfo
    {
        return $this->tools[$name] ?? null;
    }

    /**
     * @return array<string, array{FunctionInfo, array}>
     */
    public function getToolsCalled(CreateResponse $response): array
    {
        $tools = [];

        foreach (($response->choices[0]->message->toolCalls ?? []) as $toolCall) {
            $functionName = $toolCall->function->name;

            $tool = clone $this->getToolByName($functionName);
            $tool->jsonArgs = $toolCall->function->arguments;

            $tools[$toolCall->id] = $tool;
        }

        return $tools;
    }

    /**
     * @return array<string, array{FunctionInfo, array}>
     */
    public function getToolsCalledFromNode(MessageNode $messageNode): array
    {
        /** @var Message $message */
        $message = $messageNode->message;

        return $this->getToolsCalledFromMessage($message);
    }

    /**
     * @return array<string, array{FunctionInfo, array}>
     */
    public function getToolsCalledFromMessage(Message $message): array
    {
        $tools = [];

        foreach ($message->tool_calls as $toolCall) {
            $tool = clone $this->getToolByName($toolCall['function']['name']);
            $tool->jsonArgs = $toolCall['function']['arguments'];

            $tools[$toolCall['id']] = $tool;
        }

        return $tools;
    }

    public function setOpenAI(OpenAIChat $chat): void
    {
        $this->chat = $chat;

        $this->chat->setTools($this->tools);
    }
}

In the end, I had almost nothing of use from the original OpenAI wrapper this library offers. I am going into technical debt, but it bother me. It's still working. Here is my wrapper on-top of LLPhant wrapper:
<?php

declare(strict_types=1);

namespace App\LLM\Chat;

use App\LLM\Chat;
use App\LLM\Chat\Message\MessageChain;
use App\LLM\Chat\Message\MessageNode;
use App\LLM\Chat\Message\Metadata;
use App\LLM\Tools\ToolManager;
use Exception;
use LLPhant\Chat\Enums\ChatRole;
use LLPhant\Chat\FunctionInfo\FunctionInfo;
use LLPhant\Chat\FunctionInfo\FunctionRunner;
use LLPhant\Chat\Message;
use LLPhant\Chat\OpenAIChat;
use Nette\InvalidStateException;
use OpenAI\Responses\Chat\CreateResponse;

class ChatSession
{
    /** @var Chat\Observers\Observer[] */
    protected array $observers = [];

    protected ToolManager $toolManager;

    protected bool $awaitsJson = false;

    public function __construct(
        protected OpenAIChat $chat,
        protected MessageChain $messages,
        ?ToolManager $toolManager = null,
    ) {
        $this->toolManager = $toolManager ?? new ToolManager;

        $this->toolManager->setOpenAI($this->chat);
    }

    /**
     * Add an observer that will be notified when a new message is added.
     */
    public function addObserver(Chat\Observers\Observer $observer): void
    {
        $this->observers[] = $observer;
    }

    public function getObservers(string $class): array
    {
        return array_filter($this->observers, function (Chat\Observers\Observer $observer) use ($class): bool {
            return $observer instanceof $class;
        });
    }

    /**
     * Notify all registered observers about a new message.
     */
    protected function messageAdded(MessageNode $node): void
    {
        /** @var Chat\Observers\MessageObserver $observer */
        foreach ($this->getObservers(Chat\Observers\MessageObserver::class) as $observer) {
            $observer->messageAdded($node);
        }
    }

    protected function toolCalled(FunctionInfo $tool, string $toolId): void
    {
        /** @var Chat\Observers\ToolObserver $observer */
        foreach ($this->getObservers(Chat\Observers\ToolObserver::class) as $observer) {
            $observer->toolCalled($tool, $toolId);
        }
    }

    protected function toolResponded(MessageNode $node, FunctionInfo $tool, string $toolId): void
    {
        /** @var Chat\Observers\ToolObserver $observer */
        foreach ($this->getObservers(Chat\Observers\ToolObserver::class) as $observer) {
            $observer->toolResponded($node, $tool, $toolId);
        }
    }

    public function getToolManager(): ToolManager
    {
        return $this->toolManager;
    }

    /**
     * Add a message node to the chain and notify observers.
     */
    public function addMessageNode(MessageNode $node): void
    {
        $this->messages->addNode($node);

        $this->messageAdded($node);
    }

    /**
     * Main entry point for invoking a prompt and generating a response.
     *
     * @return string|array if json is set to true then return value will be decoded json content
     *
     * @throws Exception
     */
    public function __invoke(string $prompt, bool $json = false): string|array
    {
        $this->addMessageNode(new MessageNode(
            Chat\Message\Message::user($prompt, $json),
            null
        ));

        $this->awaitsJson = $json;
        $content = $this->processLastUserMessage()->message->content;

        if ($json) {
            return json_decode($content, true, flags: JSON_THROW_ON_ERROR);
        }

        return $content;
    }

    public function getMessageChain(): MessageChain
    {
        return $this->messages;
    }

    public function replaceMessageChain(MessageChain $chain): MessageChain
    {
        $old = $this->messages;

        $this->messages = $chain;

        return $old;
    }

    public function processLastUserMessage(): MessageNode
    {
        /** @var MessageNode $lastNode */
        $lastNode = $this->messages->getTail();

        if ($lastNode->message->role !== ChatRole::User) {
            throw new InvalidStateException('Cannot process last user message since, last message in chain is not issued by role User');
        }

        return $this->processMessage();
    }

    /**
     * Central method for processing messages.
     */
    private function processMessage(): MessageNode
    {
        return $this->generateResponse();
    }

    /**
     * Generates a response from the LLM using the conversation history.
     */
    private function generateResponse(): MessageNode
    {
        $start = microtime(true);

        // Enable json response
        if ($this->awaitsJson) {
            $this->chat->setModelOption('response_format', ['type' => 'json_object']);
        }

        $messageNode = $this->handleResponse(
            $this->chat->generateChatOrReturnFunctionCalled($this->messages->all()),
            $start,
            microtime(true)
        );

        if ($this->awaitsJson) {
            $this->chat->setModelOption('response_format', ['type' => 'text']);
            $this->awaitsJson = false;
        }

        return $messageNode;
    }

    /**
     * Returns the OpenAIChat instance.
     */
    public function getChat(): OpenAIChat
    {
        return $this->chat;
    }

    private function handleResponse(FunctionInfo|string $responseOrFunctionInfo, float $start, float $end): MessageNode
    {
        $messageNode = $this->processResponse($this->chat->getLastResponse(), $start, $end);
        /** @var Chat\Message\Message $message */
        $message = $messageNode->message;

        $this->addMessageNode($messageNode);

        if ($responseOrFunctionInfo instanceof FunctionInfo) {
            return $this->handleToolCalls($this->toolManager->getToolsCalledFromMessage($message));
        }

        return $messageNode;
    }

    private function handleToolCalls(array $toolCalls): MessageNode
    {
        foreach ($toolCalls as $toolId => $tool) {
            $this->toolCalled($tool, $toolId);
            $this->toolResponded($this->handleToolCall($tool, $toolId), $tool, $toolId);
        }

        return $this->generateResponse();
    }

    /**
     * Execute the tool and push back the results onto the message chain
     */
    private function handleToolCall(FunctionInfo $function, string $toolId): MessageNode
    {
        try {
            $start = microtime(true);
            $result = FunctionRunner::run($function);
            $end = microtime(true);

            // Only time we add node without processing it
            $this->addMessageNode($node = new MessageNode(
                Chat\Message\Message::toolResponse($result, $toolId),
                Metadata::fromArray(['start' => $start, 'end' => $end, 'created_at' => time()])
            ));

            return $node;
        } catch (Exception $exception) {
            return new MessageNode(
                Message::assistant("Exception occurred while executing a tool '$function->name': ".$exception->getMessage()),
                null
            );
        }
    }

    private function processResponse(CreateResponse $response, $start, $end): MessageNode
    {
        $message = Chat\Message\Message::fromResponse($response);
        $metadata = Metadata::fromResponse($response, $start, $end);

        return new MessageNode($message, $metadata);
    }
}

So this code allows me to execute prompts and be confident that JSON will be returned. It's currently highly tailored for my use case. But if you find anything inspiring, then happy to help.
$data = $chat('Generate example JSON', true);

By the way, there are some rules that your message array must adhere to. For example, you can't two user messages one after another. So for me having those Message objects stored into LinkedList made sense. I can just apply rules directly to it since I extend the LinkedList class with the MessageChain class and domain-specific methods are implemented there. Just performing a filter on the linked list where each passed node has a reference to a message before and after is quite handy.

@prykris wow that is so awesome, as someone who works in computer science; projects like these make me really excited to implement AI as Tool into code I'm working on.

@killrawr @prykris it's cool to see you guys starting using tools. In the AutoPHP code, you have some example of agentic behavior with tools.
What you want to do @killrawr could be done with AutoPHP, you just need to add the tools and it will work.
@prykris in autophp, we reuse the answer from the tool to check we have done the objective.
I'm very open to have an easier way to use tools in LLPhant


Resource limits and reservations for containers

Copied from #516
Add the ability to set the limit and reservations for cpu and memory when creating a container.

I think docker container create API doesn't support cpu reservation.
However, cpu limit, mem limit and mem reservation are here
For dev :
docker create --name testlimit --cpus=0.5 --memory=64m --memory-reservation=32m nginx

In docker API :
POST /v1.31/containers/create?name=testlimit
{
    "AttachStderr": false,
    "AttachStdin": false,
    "AttachStdout": false,
    "Cmd": null,
    "Domainname": "",
    "Entrypoint": null,
    "Env": [],
    "HostConfig": {
        "AutoRemove": false,
        "Binds": null,
        "BlkioDeviceReadBps": null,
        "BlkioDeviceReadIOps": null,
        "BlkioDeviceWriteBps": null,
        "BlkioDeviceWriteIOps": null,
        "BlkioWeight": 0,
        "BlkioWeightDevice": [],
        "CapAdd": null,
        "CapDrop": null,
        "Cgroup": "",
        "CgroupParent": "",
        "ConsoleSize": [
            0,
            0
        ],
        "ContainerIDFile": "",
        "CpuCount": 0,
        "CpuPercent": 0,
        "CpuPeriod": 0,
        "CpuQuota": 0,
        "CpuRealtimePeriod": 0,
        "CpuRealtimeRuntime": 0,
        "CpuShares": 0,
        "CpusetCpus": "",
        "CpusetMems": "",
        "DeviceCgroupRules": null,
        "Devices": [],
        "DiskQuota": 0,
        "Dns": [],
        "DnsOptions": [],
        "DnsSearch": [],
        "ExtraHosts": null,
        "GroupAdd": null,
        "IOMaximumBandwidth": 0,
        "IOMaximumIOps": 0,
        "IpcMode": "",
        "Isolation": "",
        "KernelMemory": 0,
        "Links": null,
        "LogConfig": {
            "Config": {},
            "Type": ""
        },
        "Memory": 67108864,
        "MemoryReservation": 33554432,
        "MemorySwap": 0,
        "MemorySwappiness": -1,
        "NanoCpus": 500000000,
        "NetworkMode": "default",
        "OomKillDisable": false,
        "OomScoreAdj": 0,
        "PidMode": "",
        "PidsLimit": 0,
        "PortBindings": {},
        "Privileged": false,
        "PublishAllPorts": false,
        "ReadonlyRootfs": false,
        "RestartPolicy": {
            "MaximumRetryCount": 0,
            "Name": "no"
        },
        "SecurityOpt": null,
        "ShmSize": 0,
        "UTSMode": "",
        "Ulimits": null,
        "UsernsMode": "",
        "VolumeDriver": "",
        "VolumesFrom": null
    },
    "Hostname": "",
    "Image": "nginx",
    "Labels": {},
    "NetworkingConfig": {
        "EndpointsConfig": {}
    },
    "OnBuild": null,
    "OpenStdin": false,
    "StdinOnce": false,
    "Tty": false,
    "User": "",
    "Volumes": {},
    "WorkingDir": ""
}

The --cpus flag seems to be introduced in API v1.25
Before that, you have to do it with --cpu-quota and --cpu-period
I think we have to enable cpu limit feature only for +1.25 => docker 1.13

@deviantony Can you implement storage quota limitation on UI, too?
I think many people (including me) need this feature...
e.g. --storage-opt size=1G

There are still too many caveats / constraints with that option to reliably add it into Portainer...
Sent from my iPad
On 14/11/2017, at 9:06 PM, Darron Park <notifications<EMAIL_ADDRESS>wrote:
@deviantonyhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_deviantony&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=5OQay65eU2NHfVhU46zIxGET62-GDcltcRG5TyOH6tM&s=lbovvXlB3NeV6kl2Nvyk6EGCspZbRe9vZIUl-9Sgpag&e= Can you implement storage quota limitation on UI, too?
I think many people (including me) need this feature...
e.g. --storage-opt size=1G
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_1040-23issuecomment-2D344176567&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=5OQay65eU2NHfVhU46zIxGET62-GDcltcRG5TyOH6tM&s=qyqAZc0Xr2wL7iEtMNrWy_ye9ofhBb4UW4AdWkam5SY&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlcLQrBQsKuegHu974sJjGShhscwUks5s2UnsgaJpZM4Oa42m&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=5OQay65eU2NHfVhU46zIxGET62-GDcltcRG5TyOH6tM&s=243J--jAIEX8B83N2I5IxJSkkwDbHVcWiQzTlGWbabU&e=.


git push always asking for signing even when explicitly disabled

I have checked that my configuration is not signing any commits (global/local).
Even with git push --no-signed with git commit --no-gpg-sign I get asked to sign the push.

When using lazygit, or scripts accessing performing actions on a few git repositories, I was asked to authorize the key usage to sign in (which is find and controlled by polkit), but then it asked to sign using the ssh key.
I am not 100% sure what actually helped, but I have switched back to gpgsign=true, but have not enabled ssh keys for that. I now get notified the the commit was signed, but I did not have to confirm it. If I can figure it out I will post it here.

Hi, I am still confused about the whole "SSH Key Signing Request" actions with goldwarden. I am not sure why it is asking me on every  git (pull|push|commit). All signing actions are disabled per global ~/.gitconfig and not enabled on a per project bases. What am I missing here?


docs: update installation.md

Prequisitives > Prerequisites

Describe the changes in this pull request using active verbs such as Add, Remove, Replace ...
Closes #<issue_ID>

@pradhans0906 thank you for fixing this! Can we change this in all of the version docs?


Unreliable ACC units signal

ACC units (mph/kph) [Dash_State][Units] signal changes sometimes (eg for one day every 2-3 weeks) to a different value on Global platform models. Possible solutions are finding the reason and fix or finding more reliable ACC units signal (eg using OBD2 mode22 diagnostic query) which would also simplify current ACC units logic
TODO

[ ] Implement OBD2 PID 28 query to get target market (UDM/ADM/EDM) for Global models (pre-global models do not support PID 28)
[ ] Add market and model based conditional checks
[ ] Analyze Eyesight OBD Mode22 scans to find better units bit

selfdrive/car/subaru/carstate.py

    # ACC speed units
    # Global
    #
    # metric
    # EDM 2018 Crosstrek (@mlp) metric      Dash_State Units = 3
    # EDM 2018 Crosstrek (@mlp) metric      Dash_State Units = 5
    # UDM 2019 Crosstrek metric             Dash_State Units = 4
    #
    # imperial
    # EDM 2018 Crosstrek (@mlp) imperial    Dash_State Units = 1
    # UDM 2019 Crosstrek (@Adam M) imperial Dash_State Units = 3
    #
    # Preglobal
    #
    # metric
    # ADM 2015 Outback (@Bugsy) metric:     Dash_State Units = 0
    # UDM 2016 Outback metric:              Dash_State Units = 0
    # UDM 2017 Outback (@Anthony) metric:   Dash_State Units = 0
    # UDM 2017 Legacy metric:               Dash_State Units = 1
    #
    # imperial
    # ADM 2015 Outback (@Bugsy) imperial:   Dash_State Units = x
    # UDM 2016 Outback imperial:            Dash_State Units = 1
    # UDM 2017 Outback (@Anthony) imperial: Dash_State Units = 1
    # UDM 2017 Forester (@Nougat) imperial: Dash_State Units = 0*
    # UDM 2017 Legacy imperial:             Dash_State Units = 0
    #
    # * - Preglobal Foresters do not support units change

BO_ 977 Tempo_Speed_Unit: 8 XXX
SG_ Speed_Unit_Candidate : 15|1@1+ (1,0) [0|1] "" XXX

Check this. Confirmation needed.

From my drives, I can confirm this bit is correlated with Unit setting.
On my Subaru Outback 2017 2.5 UDM
0 = Metric
1 = Imperial

Please share cabana url where you change metric / imperial a few times in dash menu. Keep comma2 camera pointed to dash while recording

I can confirm from preglobal Subaru Outback 2017 2.5 UDM cabana that there is correlation between units change in dash menu and following dbc signal
BO_ 977 Dash_State2: 8 XXX
  SG_ Units : 15|1@1+ (1,0) [0|1] "" XXX

I requested more cabana urls for different preglobal models and model years to confirm before moving forward with adding the new signal to opendbc and carstate for testing

Here is one promising candidate for Global:
BO_ 912 Dashlights: 8 XXX
 SG_ Units : 24|1@1+ (1,0) [0|1] "" XXX

Dashlights UNITS signal is confirmed to have same values on 2018 Crosstrek EDM, 2020 Crosstrek UDM and 2020 Ascent UDM

Preglobal Dash_State2 Units signal is currently confirmed working on Subaru Outback 2017 2.5 UDM and 2019 Outback UDM

There have been no reports of wrong acc speed units after new signals were upstreamed, closing this for now


Add more RBIs for Fiddle

Motivation

Add RBI for Fiddle::CParser
Add RBI for Fiddle::Importer

Test plan
See included automated tests.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_LqXA7HKXF5GXJr
→ https://go/builds/bui_LqXAzBGxo7ZIWn

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_LqXW3evjCrG41l
→ https://go/builds/bui_LqXW7rtdH8UGXX

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_LqYY5iYE5RseJB
→ https://go/builds/bui_LqYYbRWD90Jm4O


How to modify code for continuous actions?

Hello :)
I was wondering how to modify the code for continuous actions? So for example it could be compared with your naf implementation on openAI gym pendulum,
env = gym.envs.make("Pendulum-v0")
Here's how far I got,
lstm_in = 3
lstm_out = 256

class ActorCritic(nn.Module):

    def __init__(self , lstm_in ):
        super(ActorCritic, self).__init__( )

        self.lstm = nn.LSTMCell(lstm_in, lstm_out)

        self.actor_mu = nn.Linear(lstm_out, 1)
        self.actor_sigma = nn.Linear(lstm_out, 1)

        self.critic_linear = nn.Linear(lstm_out, 1)

        self.train()

    def forward(self, inputs):

        x, (hx, cx) = inputs

        #might need some RELU's here ??

        hx, cx = self.lstm(x, (hx, cx))
        x = hx

        return self.critic_linear(x), self.actor_mu(x), self.actor_sigma(x), (hx, cx)

and the code in main now looks like,
env = gym.envs.make("Pendulum-v0")
lstm_in = 3
global_model = ActorCritic( lstm_in )
global_model.share_memory()
local_model = ActorCritic( lstm_in )

It breaks with the following changes in train.py though,
env = gym.envs.make("Pendulum-v0")
s0 = env.reset()
done = True
state = torch.from_numpy(s0).float().unsqueeze(0)

value, mu, sigma, (hx, cx) = local_model((Variable(state), (hx, cx)))

#mu = mu.clamp(-1, 1) # constain to sensible values
Softplus=nn.Softplus()
sigma = Softplus(sigma) #+ 1e-5 # constrain to sensible values
normal_dist = torch.normal(mu, sigma)

prob = normal_dist
# TODO - what goes here?
#nnlog = nn.Log()
#log_prob = nnlog(prob)

#log_prob = F.log_softmax(prob)
#prob = F.softmax(logit)
#log_prob = F.log_softmax(logit)
entropy = -(log_prob * prob).sum(1)

action = prob.data
action = Variable( action )
log_prob = log_prob.gather(1, action)

#action=[0,]
state, reward, done, _ = env.step(action.data)

Any idea how to get it working?
Thanks a lot for your help,
Best regards,
Ajay
Reference - Deepmind A3C's paper, https://arxiv.org/pdf/1602.01783.pdf
Section 9 - Continuous Action Control Using the MuJoCo Physics Simulator

Picture from https://github.com/deeplearninc/relaax#distributed-a3c-architecture-with-continuous-actions

Got nearly working code in this thread,
https://discuss.pytorch.org/t/continuous-action-a3c/1033

Hi, any chance you could give me some advice? I'm still stuck trying to get this to work? Here's a post of my code
https://gist.github.com/AjayTalati/184fec867380f6fa22b9aa0951143dec
I keep getting this error,
File "main_single.py", line 174, in <module>
value_loss = value_loss + advantage.pow(2)
AttributeError: 'numpy.ndarray' object has no attribute 'pow'

I don't understand why advantage has become a numpy array instead of a torch.tensor - it never occurred with the discrete action implementation?
Any ideas what I've got wrong?
Thanks a lot for your help,
Best,
Ajay

Closing this, as continuous functions are just a pain to approximate?

I will add continuous control later. I don't have time at the moment.

OK - cool - take your time :+1:  I don't mean this as an A3C specific comment, or anything specific about your implementation.
It's just a general observation, (and perhaps a provable fact), that I've found discrete functions easier to approximate that continuous ones.
In terms of simple MLP theory, this is nice by Mhaskar and Poggio,
Learning Real and Boolean Functions: When Is Deep Better Than Shallow


whitespace - check-postbrace being flagged by IDE, and not in configs/all.ts

Bug Report

TSLint version: 5.12.1
TypeScript version: 2.6.2
Running TSLint via:  WebStorm

Actual behavior
My IDE (WebStorm) complains that check-postbrace isn't a valid option for the whitespace rule in tslint.json, and suggests I replace it with one of a list of options: ["check-branch", "check-decl", "check-operator", "check-module", "check-separator", "check-rest-spread", "check-type", "check-typecast", "check-type-operator", "check-preblock"]
Tslint runs properly, however, and the rule works
Expected behavior
check-postbrace should not be showing a warning about not being a valid value.
Notes
I'm not sure if this is an IDE issue, or if it's something inside of tslint. https://github.com/palantir/tslint/blob/master/src/configs/all.ts doesn't list check-postbrace as a valid option under whitespace, so maybe it's just missing from some enum somewhere?

@fushi your suspicion of this being external might be correct. What version of TSLint is WebStorm running?
check-postbrace was added in https://github.com/palantir/tslint/pull/4068, which was only recently released. Perhaps it's on an old version?

I'm running PHPStorm 2018.3, TS 3.3.1 and tslint 5.12.1 but I'm still seeing this error.

missed in WebStorm 2018.3.1\plugins\tslint\lib\tslint.jar!\tslintJsonSchema\tslint-schema.json

☠ TSLint's time has come! ☠
TSLint is no longer accepting most feature requests per #4534. See typescript-eslint.io for the new, shiny way to lint your TypeScript code with ESLint. ✨
It was a pleasure open sourcing with you all!

🤖 Beep boop! 👉  TSLint is deprecated 👈 (#4534)  and you should switch to typescript-eslint! 🤖
🔒 This issue is being locked to prevent further unnecessary discussions. Thank you! 👋


Fix Angular2-Meteor NPM for Windows

Original source of the issue is here:
https://github.com/Urigo/angular2-meteor/issues/171#issuecomment-199806027

any workaround until this is fixed ?

https://cygwin.com might help.

Ok, I added some temporary fix. Please, check that it works for you.

npm install https://github.com/Urigo/angular2-meteor --save

><EMAIL_ADDRESS>install ..\GIT\Meteor\webapp\node_modules\angular2-meteor
> node install.js

module.js:327
    throw err;
    ^

Error: Cannot find module 'fs.extra'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (..\GIT\Meteor\webapp\node_modules\angular2-meteor\install.js:1:72)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
<EMAIL_ADDRESS>..\GIT\Meteor\webapp
`--<EMAIL_ADDRESS>
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>No repository field.
npm WARN<EMAIL_ADDRESS>No license field.
npm ERR! Windows_NT 10.0.10586
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\Maik Laschober\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "install" "https://github.com/Urigo/angular2-meteor" "--save"
npm ERR! node v4.3.1
npm ERR! npm  v3.8.2
npm ERR! code ELIFECYCLE

npm ERR<EMAIL_ADDRESS>install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script 'node install.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the angular2-meteor package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node install.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs angular2-meteor
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls angular2-meteor
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     ..\GIT\Meteor\webapp\npm-debug.log

mv on windows is typically move command

Ok, there was some minor correction you might have missed. What would be if you try once more?

seems to work now
+--<EMAIL_ADDRESS>(git+https://github.com/Urigo/angular2-meteor.git#c29821a8256532df18f235c55e8127a16da6ea1f)
| +--<EMAIL_ADDRESS>| | `--<EMAIL_ADDRESS>| `--<EMAIL_ADDRESS>|   +--<EMAIL_ADDRESS>|   | +--<EMAIL_ADDRESS>|   | +--<EMAIL_ADDRESS>|   | `--<EMAIL_ADDRESS>|   +--<EMAIL_ADDRESS>|   `--<EMAIL_ADDRESS>|     `--<EMAIL_ADDRESS>

not working on the bootstrap npm

what does it mean the bootstrap npm? does npm install work now?

pardon,
i meant:
npm install https://github.com/Urigo/angular2-meteor --save // this works fine
npm install https://github.com/Urigo/angular2-meteor-auto-bootstrap --save // this didn't work (mv still as issue)

i'am at home now will try it again tomorrow morning.

I am having this same issue with mv command on Windows for the angular2-meteor-auto-bootstrap package. Is there a solution without using cygwin before tomorrow?

Being at work again, npm install https://github.com/Urigo/angular2-meteor-auto-bootstrap --save still not working on windows

Ok, I committed fix (https://github.com/Urigo/angular2-meteor-auto-bootstrap/issues/4), will be available soon. Closing this one.

@Devidian can you confirm it works for you?

Currently on easter vacation. Checking on tuesday

its working fine ;)


Added -q (quiet) option to push, popd, dirs functions.

Resolves issue #753.
No unit tests added yet; I'm not familiar with the framework, and not sure how best to do them.

@nfischer Okay, have a look now, I think things should be in order. :-)

Oh, this needs docs generated. Please run npm run gendocs

Mostly good.
I don't have a strong opposition to dirs -q. It makes logical sense compared to pushd/popd. If dirs ever does add a -q flag which differs in behavior, then we'll have to change. @freitagbr thoughts?

Yep, those were my thoughts too.

That's all done now. Word of warning: the existing code for pushd and popd writes to stderr, which differs in behaviour from the shell builtins for bash & zsh, from what I can tell. I've left it writing to stderr, but you may want to change that. (I don't know what function you'd want to call other that common.log, if so...)

the existing code for pushd and popd writes to stderr, which differs in behaviour from the shell builtins

Yup, you're right. This is unfortunate legacy behavior. Feel free to file a bug and we'll get to it for a breaking release.

It prints before the mock is initialised though. Isn’t that aufficient?
Sent from my iPhone

On 9 Oct 2017, at 04:19, Nate Fischer<EMAIL_ADDRESS>wrote:
@nfischer requested changes on this pull request.
LGTM % comment. Thanks!
In test/popd.js:

try {
shell.config.silent = false;

shell.pushd('test/resources/pushd');
I think this should be moved up one line, otherwise it will print during the test (which we try to avoid). Same thing for the quite mode on test.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

It prints before the mock is initialised though. Isn’t that aufficient?

You're correct: it doesn't affect correctness, but test results would be difficult to read if we let integration tests print to the console. That's why we use shell.config.silent in the first place during our tests.

No problem. That's done now.

Codecov Report

Merging #777 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #777      +/-   ##
==========================================
+ Coverage   95.39%   95.39%   +<.01%
==========================================
  Files          33       33
  Lines        1237     1239       +2
==========================================
+ Hits         1180     1182       +2
  Misses         57       57

Impacted Files
Coverage Δ

src/dirs.js
97.18% <100%> (+0.08%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c889075...2168ee5. Read the comment docs.


I need some help with calibration or using already existing calibration data

Hello. I have a device with a set of IR cameras in stereo configuration and a built-in IMU. Unfortunately this is a completely proprietary device, so there is no documentation for it whatsoever.
I can access the camera feeds form OpenCV, and the IMU data too, but I don't know how to get the calibration data required to integrate Kimera-VIO with this device. I was trying to use kalibir in the past, but I didn't find a comprehensive guide that shows me exactly what steps I need to take to calibrate my device, if I don't have any kalibir data files to start with.
I have managed to pull some calibration data from the firmware, which might be useful:
I have an IMU calibration file, that seems to contain the IMU transform, and some rectification data. Applying the transform, rectification and offset data to the raw IMU values is relatively easy.
But here's the more important file, the Camera calibration file. This one contains the sensor model (OV7251) along with a ton of calibration data like Projection and Distortion. Unfortunately this file is proprietary, so I don't have any documentation for it, but I'm hoping that the VIO experts working on this project can help me decode it, and potentially even use it instead of having to use kalibir.
The first five data fields in this file are obvious. The parts that I'm unable to decode are "Projection" and "Distortion". There are some model names written in there (PinholeSymmetric and Fisheye62), but I don't exactly know what they mean.
My goal is to be able to use Kimera-VIO to accurately track the device's movement in real time (at IMU update rate) with no (or minimal) latency. I don't need mapping, or 3D reconstruction, only the accurate movement tracking.
There are five cameras on the device, but I only intend to use the two forward-facing ones with Kimera-VIO, at least for now.
Any bit of help is very much appreciated.

better late than never: the Rift-S "fisheye62" is documented here now https://github.com/mapillary/OpenSfM/blob/874b9114f854d9c664b06bc187384934ce5d92a0/opensfm/src/geometry/src/camera.cc#L69 and SLAM is now available in Monado for it https://monado.freedesktop.org/ (thanks to @thaytan)


Separate GATTS and GATTC events (IDFGH-6897)

Hi Espressif,
I have un suggestion about the event's management in Bluedroid.
Context:
I'm using the IDF 4.4 with bluedroid enable.
In my application, I'm using the ESP32 (in client mode) to connect to another iot device in BLE and also a smartphone can be connected to the ESP32 (in server mode).
Can be tested with gattc_gatts_coex example.
Problem:
When an event appears (like CONNECTED or DISCONNECTED), I received it in gattc event handler and in gatts event handler without taking into account where the event come from.
Idea:
Received the right event only in the right handler.
Thank you and best regards,

@SamyRICHET Thanks for your suggestions. We have no plans to separate the GATTS and GATTC events at this time. You can process these events according to remote address.

Thanks for reporting, feel free to reopen.


fix '_lseeki64' undefined compiler warning

Thanks to work done by @gengjiawen (https://github.com/nodejs/node/pull/31153) we're now running Node.js tests as GitHub actions and this appears to be annotating pull requests in core with build warnings from unchanged files, one of which is from deps/uvwasi/src/uvwasi.c:

The full warning, from https://github.com/cjihrig/uvwasi/runs/362355578#step:5:383:
##[warning]D:\a\uvwasi\uvwasi\src\uvwasi.c(225,16): warning C4013: '_lseeki64' undefined; assuming extern returning int [D:\a\uvwasi\uvwasi\uvwasi_a.vcxproj]

Thanks!


add command to view details of an item

also changed the UG link

Codecov Report

Merging #44 into master will decrease coverage by 0.65%.
The diff coverage is 42.02%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc9e99c...8a6afa0. Read the comment docs.


Customize github api url

Pre-Flight checklist

[ ] Testing

[ ] Code is tested
[ ] Changes have been manually tested

[x] Settings

[x] New settings are documented and present in app.json
[x] New settings have reasonable development defaults, if applicable
[ ] Opened issue for DevOps regarding necessary configuration changes to deployed environments

What are the relevant tickets?
Closes https://github.com/mitodl/ocw-studio/issues/238
What's this PR do?
Makes the github API customizable
How should this be manually tested?
(Required)
Where should the reviewer start?
(Optional)
Any background context you want to provide?
(Optional)
Screenshots (if appropriate)
(Optional)
What GIF best describes this PR or how it makes you feel?
(Optional)

Codecov Report

Merging #239 (2232fe1) into master (a18036f) will increase coverage by 5.30%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #239      +/-   ##
==========================================
+ Coverage   87.19%   92.49%   +5.30%
==========================================
  Files         117       57      -60
  Lines        3092     1213    -1879
  Branches      472      271     -201
==========================================
- Hits         2696     1122    -1574
+ Misses        359       90     -269
+ Partials       37        1      -36

Impacted Files
Coverage Δ

repl.py

content_sync/tasks.py

websites/config_schema/api.py

ocw_import/conftest.py

news/views.py

websites/constants.py

localdev/configs/api.py

main/admin.py

main/features.py

content_sync/api.py

... and 49 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a18036f...2232fe1. Read the comment docs.

@rhysyngsun ready for another look


Add session management

[x] Create new session objects
[x] Allow integration with Django sessions
[x] Add tests for sessions
[x] Update documentation

Closes #372

@gunthercox will it consider other bot session objects, for example gitter/microsoft session object was expired then chatterbot session object will expire?, will this case is considered?

If two bots are created, each will have sessions separate from each other.
bot1 = ChatBot()
bot2 = ChatBot()

For each bot, the sessions could be shared across multiple adapters as long as there is a shared session id. Right now this is something that the developer would have to handle manually because a ChatBot instance has no way of knowing that a Gitter user is the same person as a user on the Microsoft Bot Framework. To handle this, the developer would likely have to store associations between a unique identifier and a user's verified usernames for each service (Gitter, HipChat, Microsoft Bot Framework, etc.).
Hopefully this answers your question. I'm planning to add more documentation on this to hopefully make it clearer.

Nice, Thank you


Go template sprig date functions not defined

What happened:
I got the error <function "now" not defined> when running syft with a Go template that is using the "now" function, as explained on http://masterminds.github.io/sprig/date.html.
What you expected to happen:
I expected to be able to use sprig date functions.
It seems that "now", "date", "dateInZone" are not found, but "ago", "duration", "durationRound", "unixEpoch" are found.
Steps to reproduce the issue:
sbom.go.tmpl:
{{ now | unixEpoch }}
-> syft . -o template=sbom -t sbom.go.tmpl
✔ Indexed file system                                                                                .
✔ Cataloged packages              [0 packages]
[0000]  WARN no explicit name and version provided for directory source, deriving artifact ID from the given path (which is not ideal)
[0000]  WARN cataloger failed cataloger=sbom-cataloger error=sbom format not recognized location=/sbom
1 error occurred:
* failed to write SBOM: 1 error occurred:
* unable to write SBOM: unable to parse template: template: sbom.go.tmpl:1: function "now" not defined
Anything else we need to know?:
Environment:

Output of syft version:
Application: syft
Version:    0.97.1
BuildDate:  2023-11-17T20:53:01Z
GitCommit:  Homebrew
GitDescription: [not provided]
Platform:   darwin/arm64
GoVersion:  go1.21.4
Compiler:   gc
OS (e.g: cat /etc/os-release or similar):
sw_vers
ProductName:		macOS
ProductVersion:		14.1.1
BuildVersion:		23B81

Hi @david-gherghita, thanks for the report. There is a similar discussion for Grype's templating capability here: https://github.com/anchore/grype/issues/1243 -- it looks like the outcome is that we decided not to include the Sprig non-hermetic functions. I can run it by the team, but I believe we would come to the same conclusion for Syft. Can you describe in a bit more detail what you're trying to do, and maybe we can suggest an alternative?

Hello @tgerla, thank you very much for your prompt response. I understand the situation.
What I am trying to achieve is to generate a slightly modified version of the CycloneDX 1.4 SBOM than you currently offer with the included format. For this, I need to also include the timestamp.
I noticed that for Grype the timestamp was included as a property of "Document.Descriptor": "Document.Descriptor.Timestamp". Do you think something like that could be done for syft too?

It looks like that cyclonedx documents already include a timestamp:
❯ syft alpine:3.12 -o<EMAIL_ADDRESS>| jq '.metadata'
 ✔ Loaded image                                                                                alpine:3.12
 ✔ Parsed image                                                                                sha256:cc604a625da1289c5dd57f947318133161ff7f40fb03dc2a649300473b97e743
 ✔ Cataloged packages              [15 packages]
{
  "timestamp": "2023-11-30T16:11:11-05:00",
  "tools": [
    {
      "vendor": "anchore",
      "name": "syft",
      "version": "0.98.0"
    }
  ],
  "component": {
    "bom-ref": "b0b5eff358e3ab6d",
    "type": "container",
    "name": "alpine",
    "version": "3.12"
  }
}

Is there another timestamp section that you're referring to?

@david-gherghita
We probably won't add the full set of non-hermetic functions to the template options, but we could add select now and date functions... would that be helpful?

Hello @wagoodman,
That's exactly the timestamp I need, but I want to use it in my own output file, generated from a Go template. And so far I couldn't find a way to integrate the timestamp into the template.
These two functions now and date would fully meet my requirements; implementing them would resolve my issue.
Thank you.

Hello, I would like to bump this issue..
I am also planning to make a more minimal SBOM and for that I need a timestamp and when possible the uuid for the serial number field of the output SBOM. (for example via the now and uuidv4 function of sprig).
A timestamp and uuid/serialNumber field in the descriptor of the SBOM would be sufficient :)

Bump


blockchain: Add invalidateblock rpc call

This PR adds the missing invalidateblock rpc call. The call is useful for both manually testing changes with the cli or creating automated tests.
The behavior of Bitcoin Core is mimicked. However, since we currently don't keep track of which block was received first, the prioritization of choosing whichever block is received first for the active chain tip is omitted.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

77 of 90 (85.56%)  changed or added relevant lines in 2 files are covered.
11 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.08%) to 53.382%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

blockchain/chain.go
71
84
84.52%

Files with Coverage Reduction
New Missed Lines
%

connmgr/connmanager.go
2
86.07%

peer/peer.go
2
75.45%

btcec/signature.go
3
92.76%

database/ffldb/blockio.go
4
93.68%

Totals

Change from base Build<PHONE_NUMBER>:
0.08%

Covered Lines:
27473

Relevant Lines:
51465

💛  - Coveralls


Update STEPS.md

Removing unnecessary text in docs.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Site Reorg Follow-up tasks

These tasks are remaining after work done for the initial site reorg, as discussed in https://github.com/dendronhq/dendron/discussions/1665

[ ] Work on tagged with #todovault/dendron.user-guide.basics.md:19:- Journals #todo
vault/dendron.user-guide.editing.md:11:- Math and Diagrams #todo
vault/dendron.user-guide.extending.md:10:- Note Type System #todo
vault/dendron.user-guide.extending.md:12:- Custom Pods #todo
vault/dendron.user-guide.organizing.md:14:    - Journals #todo
vault/dendron.user-guide.retrieving.md:14:- Search #todo

[ ] Update schemas for the site reorg: We need to ensure paths are updated in schema files to adhere to the new layout
[ ] Merge [[dendron._ref.terms]] with [[dendron.concepts]]
[ ] Organize tags
[ ] Organize by area (areas are children of [[dendron.user-guide]])

[[dendron.faq]]
[[dendron.concepts]]
[[dendron.guides.best-practices]]
[[dendron.guides.tips]]

[ ] Clean up [[dendron.ref.cli]]
[ ] Clean up [[dendron.ref.config]]
[x] Organize [[community]]

More info
For organizing, if you look at FAQ or CLI, they are either random or alphabetically ordered. We want to order them the same way we do our user guide

Basics: fundamental core features of Dendron
Editing
Retrieving
Organizing
Sharing
Transferring: move notes to other places (pods)
Extending: extending Dendron (note type system, custom pods, hooks)

Comments from the community discord, to confirm these are resolved upon completion of reorg:

I also think that the documentation at https://wiki.dendron.so/ could be improved to resolve the following issues

If you go to https://wiki.dendron.so/ and only look at the content in the left sidebar, there seems to be no entry for Publishing at all (unless I'm being blind. That's possible). I only found the publishing page at https://wiki.dendron.so/notes/4ushYTDoX0TYQ1FDtGQSg/ by dumb luck, because I already knew that Dendron supported publishing. There seems to be no way to get to https://wiki.dendron.so/notes/4ushYTDoX0TYQ1FDtGQSg/ from the left sidebar at https://wiki.dendron.so/
I then noticed that a lot of useful stuff is at https://wiki.dendron.so/notes/7a7af2fa-b9d0-4adc-81e6-f03186123539/ . However, there seems to be no way to get to that page from the left sidebar at https://wiki.dendron.so/
The left side bar at https://wiki.dendron.so/ has 'User Guide', 'Guide', 'Concepts' as top level menu entries. It is not immediately clear what the differences are. E.g. What is the difference between 'User Guide' and 'Guide' ? These could be all merged under 'User Guide' or could have more descriptive titles. Also, I'd have expected that at least one of these have 'Publishing' as a sub-link in the left bar, but they do not.

@ScriptAutomate Are there PRs for the tasks already? Let me know if I can pick something as a contribution.

Are there PRs for the tasks already? Let me know if I can pick something as a contribution.

@hrmnjt Hey! If still interested, would you like to take a look at the  task:

Merge [[dendron._ref.terms]] with [[dendron.concepts]]

It would be a great one to start with. It's pretty much:

Create a new, local branch from dev in a fork of this repo
Run Move Header command for each header in dendron._ref.terms to dendron.concepts
For Command Palette, since it is already in dendron.concepts, simply manually/temporarily rename the existing Command Palette header in dendron.concepts (don't use Rename Header for this) before doing Move Header of Command Palette from dendron._ref.terms. Then review and combine the contents of both of those headers in dendron.concepts again.
Then Delete Node on dendron._ref.terms. This should show, after doing the above steps, that no remaining notes are impacted
Submit PR against dev branch

Great. Let me check out dendron._ref and raise PR


Fix to compile with scala-2.11

cc @andypetrella

scala 2.11 is failing because of crappy test which is fixed  in #466


Use legacy console option may prevent Terminal from launching ConPTYs correctly

I got this issue and after uncheck Use legacy console option, my terminal show up

Originally posted by @oclockvn in https://github.com/microsoft/terminal/issues/1364#issuecomment-508796240
This represents double-checking that the startup routines in entrypoints.cpp are ignoring the v1/v2 status when identifying a ConPTY-style connection and always doing v2.

Confirming: we check the ForceV1 key and fail out if the user has ForceV1 and is requesting commandline launch. To make legacy console not interfere with CreatePseudoConsole (which doesn't use commandline launch), we need to detect that the server handle being passed in is for pseudoconsole work.
We could key off the presence of --signal in both cases.

(Confirmed this is broken in the windows image as well. pwincon refuses to launch because it bombs out in CreatePseudoConsole :))


discover bug >=2.19.2

Expected behaviour:
Successfully installed flexget-2.19.1
root@cloudBox:~# flexget execute --now --discover-now
2019-02-08 04:04 VERBOSE  task_queue                    There are 21 tasks to execute. Shutdown will commence when they have completed.
2019-02-08 04:04 VERBOSE  input_cache   Remove_Canceled Restored 6 entries from db cache
2019-02-08 04:04 VERBOSE  details       Remove_Canceled Produced 6 entries.
2019-02-08 04:04 VERBOSE  details       Remove_Canceled Summary - Accepted: 0 (Rejected: 0 Undecided: 6 Failed: 0)

Actual behaviour:
Successfully installed flexget-2.19.2
root@cloudBox:~# flexget execute --now --discover-now
2019-02-08 04:04 CRITICAL plugin                        Plugin `flexget.plugins.SeriesSearch` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/home/snikay/.flexget/plugins/SeriesSearch.py", line 7, in <module>
    from flexget.utils.search import normalize_unicode
ImportError: No module named search
2019-02-08 04:04 CRITICAL plugin                        Plugin `flexget.plugins.MoviesSearch` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/home/snikay/.flexget/plugins/MoviesSearch.py", line 7, in <module>
    from .BaseSearchPlugin import BaseSearchPlugin, SearchResultEntry
  File "/home/snikay/.flexget/plugins/BaseSearchPlugin.py", line 7, in <module>
    from flexget.utils.search import normalize_unicode
ImportError: No module named search
2019-02-08 04:04 CRITICAL plugin                        Plugin `flexget.plugins.BaseSearchPlugin` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/home/snikay/.flexget/plugins/BaseSearchPlugin.py", line 7, in <module>
    from flexget.utils.search import normalize_unicode
ImportError: No module named search
2019-02-08 04:04 CRITICAL plugin                        Plugin `flexget.components.managed_lists.lists.entry_list.cli` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/usr/local/lib/python2.7/dist-packages/flexget/components/managed_lists/lists/entry_list/cli.py", line 13, in <module>
    from . import db
  File "/usr/local/lib/python2.7/dist-packages/flexget/components/managed_lists/lists/entry_list/db.py", line 4, in <module>
    from typing import MutableSet
ImportError: No module named typing
2019-02-08 04:04 CRITICAL plugin                        Plugin `flexget.components.managed_lists.lists.entry_list.api` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/usr/local/lib/python2.7/dist-packages/flexget/components/managed_lists/lists/entry_list/api.py", line 21, in <module>
    from . import db
  File "/usr/local/lib/python2.7/dist-packages/flexget/components/managed_lists/lists/entry_list/db.py", line 4, in <module>
    from typing import MutableSet
ImportError: No module named typing
2019-02-08 04:04 CRITICAL plugin                        Plugin `flexget.components.managed_lists.lists.entry_list.db` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/usr/local/lib/python2.7/dist-packages/flexget/components/managed_lists/lists/entry_list/db.py", line 4, in <module>
    from typing import MutableSet
ImportError: No module named typing
2019-02-08 04:04 CRITICAL plugin                        Plugin `flexget.components.managed_lists.lists.entry_list.entry_list` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/usr/local/lib/python2.7/dist-packages/flexget/components/managed_lists/lists/entry_list/entry_list.py", line 12, in <module>
    from . import db
  File "/usr/local/lib/python2.7/dist-packages/flexget/components/managed_lists/lists/entry_list/db.py", line 4, in <module>
    from typing import MutableSet
ImportError: No module named typing
2019-02-08 04:04 CRITICAL manager                       [/templates/Discover-enable-Backfill/discover/from/0] The key `searchSerienjunkies` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       [/templates/Discover-disable-Backfill/discover/from/0] The key `searchSerienjunkies` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Main/discover/from/0] The key `searchHdarea` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Main/discover/from/1] The key `searchHdworld` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Main/discover/from/2] The key `searchMovieBlog` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Low/discover/from/0] The key `searchHdarea` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Low/discover/from/1] The key `searchHdworld` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Low/discover/from/2] The key `searchMovieBlog` is not valid here. Only known plugin names are valid keys.
2019-02-08 04:04 CRITICAL manager                       Failed to load config file: Did not pass schema validation.
Could not start manager: Did not pass schema validation.
root@cloudBox:~# sudo pip install --upgrade flexget==2.19.1
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
The directory '/home/snikay/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/snikay/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting flexget==2.19.1
  Downloading https://files.pythonhosted.org/packages/4a/04/fa9d1efdd500a8bb57b799ea374800054110d189499f1390fb5029a3bf44/FlexGet-2.19.1-py2.py3-none-any.whl (6.4MB)
    100% |████████████████████████████████| 6.5MB 6.3MB/s
Requirement already satisfied, skipping upgrade: future==0.16.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.16.0)
Requirement already satisfied, skipping upgrade: flask-login==0.4.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.4.0)
Requirement already satisfied, skipping upgrade: pytz==2017.2 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2017.2)
Requirement already satisfied, skipping upgrade: certifi==2017.4.17 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2017.4.17)
Requirement already satisfied, skipping upgrade: pyparsing==2.2.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.2.0)
Requirement already satisfied, skipping upgrade: progressbar==2.5 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.5)
Requirement already satisfied, skipping upgrade: cherrypy==10.2.2 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (10.2.2)
Requirement already satisfied, skipping upgrade: guessit==3.0.3 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (3.0.3)
Requirement already satisfied, skipping upgrade: html5lib==0.999999999 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.999999999)
Requirement already satisfied, skipping upgrade: importlib-metadata==0.8 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.8)
Requirement already satisfied, skipping upgrade: pynzb==0.1.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.1.0)
Requirement already satisfied, skipping upgrade: flask==1.0.2 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.0.2)
Requirement already satisfied, skipping upgrade: colorclass==2.2.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.2.0)
Requirement already satisfied, skipping upgrade: itsdangerous==0.24 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.24)
Requirement already satisfied, skipping upgrade: aniso8601==1.2.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.2.1)
Requirement already satisfied, skipping upgrade: click==6.7 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (6.7)
Requirement already satisfied, skipping upgrade: rebulk==0.9.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.9.0)
Requirement already satisfied, skipping upgrade: six==1.10.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.10.0)
Requirement already satisfied, skipping upgrade: feedparser==5.2.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (5.2.1)
Requirement already satisfied, skipping upgrade: path.py==11.5.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (11.5.0)
Requirement already satisfied, skipping upgrade: webencodings==0.5.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.5.1)
Requirement already satisfied, skipping upgrade: pyrss2gen==1.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.1)
Requirement already satisfied, skipping upgrade: flask-restful==0.3.6 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.3.6)
Requirement already satisfied, skipping upgrade: flask-cors==3.0.2 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (3.0.2)
Requirement already satisfied, skipping upgrade: werkzeug==0.14.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.14.1)
Requirement already satisfied, skipping upgrade: terminaltables==3.1.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (3.1.0)
Requirement already satisfied, skipping upgrade: tzlocal==1.4 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.4)
Requirement already satisfied, skipping upgrade: apscheduler==3.5.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (3.5.0)
Requirement already satisfied, skipping upgrade: babelfish==0.5.5 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.5.5)
Requirement already satisfied, skipping upgrade: plumbum==1.6.3 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.6.3)
Requirement already satisfied, skipping upgrade: tempora==1.8 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.8)
Requirement already satisfied, skipping upgrade: markupsafe==1.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.0)
Requirement already satisfied, skipping upgrade: sqlalchemy==1.2.6 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.2.6)
Requirement already satisfied, skipping upgrade: python-dateutil==2.6.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.6.1)
Requirement already satisfied, skipping upgrade: zipp==0.3.3 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.3.3)
Requirement already satisfied, skipping upgrade: pyyaml==3.13 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (3.13)
Requirement already satisfied, skipping upgrade: cheroot==5.5.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (5.5.0)
Requirement already satisfied, skipping upgrade: jinja2==2.10 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.10)
Requirement already satisfied, skipping upgrade: jsonschema==2.6.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.6.0)
Requirement already satisfied, skipping upgrade: flask-restplus==0.10.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (0.10.1)
Requirement already satisfied, skipping upgrade: requests==2.20.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.20.1)
Requirement already satisfied, skipping upgrade: portend==1.8 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.8)
Requirement already satisfied, skipping upgrade: chardet==3.0.3 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (3.0.3)
Requirement already satisfied, skipping upgrade: urllib3==1.21.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.21.1)
Requirement already satisfied, skipping upgrade: rpyc==4.0.1 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (4.0.1)
Requirement already satisfied, skipping upgrade: flask-compress==1.4.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (1.4.0)
Requirement already satisfied, skipping upgrade: idna==2.5 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (2.5)
Requirement already satisfied, skipping upgrade: beautifulsoup4==4.6.0 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (4.6.0)
Requirement already satisfied, skipping upgrade: zxcvbn-python==4.4.15 in /usr/local/lib/python2.7/dist-packages (from flexget==2.19.1) (4.4.15)
Requirement already satisfied, skipping upgrade: setuptools>=18.5 in /usr/local/lib/python2.7/dist-packages (from html5lib==0.999999999->flexget==2.19.1) (40.8.0)
Requirement already satisfied, skipping upgrade: configparser; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from importlib-metadata==0.8->flexget==2.19.1) (3.5.0)
Requirement already satisfied, skipping upgrade: contextlib2; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from importlib-metadata==0.8->flexget==2.19.1) (0.5.5)
Requirement already satisfied, skipping upgrade: pathlib2; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from importlib-metadata==0.8->flexget==2.19.1) (2.3.3)
Requirement already satisfied, skipping upgrade: backports.os; python_version == "2.7" and sys_platform == "linux2" in /usr/local/lib/python2.7/dist-packages (from path.py==11.5.0->flexget==2.19.1) (0.1.1)
Requirement already satisfied, skipping upgrade: funcsigs; python_version == "2.7" in /usr/local/lib/python2.7/dist-packages (from apscheduler==3.5.0->flexget==2.19.1) (1.0.2)
Requirement already satisfied, skipping upgrade: futures; python_version == "2.7" in /usr/local/lib/python2.7/dist-packages (from apscheduler==3.5.0->flexget==2.19.1) (3.2.0)
Requirement already satisfied, skipping upgrade: functools32; python_version == "2.7" in /usr/local/lib/python2.7/dist-packages (from jsonschema==2.6.0->flexget==2.19.1) (3.2.3.post2)
Requirement already satisfied, skipping upgrade: scandir; python_version < "3.5" in /usr/local/lib/python2.7/dist-packages (from pathlib2; python_version < "3"->importlib-metadata==0.8->flexget==2.19.1) (1.9.0)
Installing collected packages: flexget
  Found existing installation: FlexGet 2.19.2
    Uninstalling FlexGet-2.19.2:
      Successfully uninstalled FlexGet-2.19.2
Successfully installed flexget-2.19.1
root@cloudBox:~# flexget execute --now --discover-now
2019-02-08 04:04 VERBOSE  task_queue                    There are 21 tasks to execute. Shutdown will commence when they have completed.
2019-02-08 04:04 VERBOSE  input_cache   Remove_Canceled Restored 6 entries from db cache
2019-02-08 04:04 VERBOSE  details       Remove_Canceled Produced 6 entries.
2019-02-08 04:04 VERBOSE  details       Remove_Canceled Summary - Accepted: 0 (Rejected: 0 Undecided: 6 Failed: 0)

Steps to reproduce:
External scripts with discover
Config:
https://github.com/crack00r/Flexget-Config

This was fixed in 2.19.3

still happening in x.5
2019-02-10 00:02 CRITICAL plugin                        Plugin `flexget.plugins.SeriesSearch` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/home/snikay/.flexget/plugins/SeriesSearch.py", line 7, in <module>
    from flexget.utils.search import normalize_unicode
ImportError: No module named search
2019-02-10 00:02 CRITICAL plugin                        Plugin `flexget.plugins.MoviesSearch` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/home/snikay/.flexget/plugins/MoviesSearch.py", line 7, in <module>
    from .BaseSearchPlugin import BaseSearchPlugin, SearchResultEntry
  File "/home/snikay/.flexget/plugins/BaseSearchPlugin.py", line 7, in <module>
    from flexget.utils.search import normalize_unicode
ImportError: No module named search
2019-02-10 00:02 CRITICAL plugin                        Plugin `flexget.plugins.BaseSearchPlugin` failed to import dependencies
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 392, in _import_plugin
    __import__(module_name)
  File "/home/snikay/.flexget/plugins/BaseSearchPlugin.py", line 7, in <module>
    from flexget.utils.search import normalize_unicode
ImportError: No module named search
2019-02-10 00:02 CRITICAL manager                       [/templates/Discover-enable-Backfill/discover/from/0] The key `searchSerienjunkies` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       [/templates/Discover-disable-Backfill/discover/from/0] The key `searchSerienjunkies` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Main/discover/from/0] The key `searchHdarea` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Main/discover/from/1] The key `searchHdworld` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Main/discover/from/2] The key `searchMovieBlog` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Low/discover/from/0] The key `searchHdarea` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Low/discover/from/1] The key `searchHdworld` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       [/tasks/Movies_Watchlist_Search_DE_Low/discover/from/2] The key `searchMovieBlog` is not valid here. Only known plugin names are valid keys.
2019-02-10 00:02 CRITICAL manager                       Failed to load config file: Did not pass schema validation.
Could not start manager: Did not pass schema validation.
snikay@cloudBox:/etc/systemd/system$ flexget -V
2.19.5
You are on the latest release.

@snickers2k Looks like your issue is from some custom plugins. We moved one of the imports they are using and they'll need updated. from flexget.components.sites.utils import normalize_unicode


[macOS] Automatically detect MoltenVK libs installed via homebrew and macports.

Adds more options for MoltenVK auto-detection as well as allow setting vulkan_sdk_path which is pointing at libMoltenVK.a instead of .xcframework.

Thanks!


Newsletters: add 251 (2023-05-17)

[x] News and PRs @harding
[ ] Lede, releases/RCs, and topic entries @harding
[x] Why do we have a mempool? @glozow
[x] PR to be assigned @Xekyo
[x] Suggestion: Coredev meeting summary notes (https://github.com/bitcointranscripts/bitcointranscripts/pull/247)

Made edits for all feedback, except some minor comments about the mempool section.  Updated releases/RCs and added topic entries.  Thanks everyone for the contributions and feedback!

Corrected a link in the LSP section. Made a small change to the mempool writeup.
Thanks @harding @glozow @Xekyo @sipa @LarryRuane @vostrnad @ClaraShk for your contributions this week!


Update to Serde 1.0

Serde 0.9.0 release notes: https://github.com/serde-rs/serde/releases/tag/v0.9.0
Serde 1.0.0 release notes: https://github.com/serde-rs/serde/releases/tag/v1.0.0

@laktak Any information on the status of this?

@jkcclemens this will probably take a while. PRs are welcome if there are any takers.

I'll close this issue in favor of #6


Send appropriate JSON content header

Recommend switch from res.send to res.json in order to ensure appropriate JSON content header -- preferable for integration and various dev tools.

Merged


expand data request

Added a computation of the data volume to the data request, and updated a few details.
The computation should render as

The switch to .qmd is necessary for quarto to render the folding code correctly.

I would suggest that this be stamped 'DRAFT Data Request' at the top until an infrastructure group (and maybe the steering group) has discussed it.
Okay, so we are going to ask for HealPix level 9 (13km)? Not level 10 (6km, closer to the native grids of many models). Why 13km? To make it smaller?
You are also asking for zarr and intake-esm catalogs. I think many will need some help with that. I will certainly. Also you want interpolation to standard levels. I was going to leave data on the native levels.
Also, what compression is going to be used? I'm not a big fan of compression that slows down the reading (which most of it has seemed to do), but I'm open to specific suggestions.
One question: will HealPix work with a non-global grid? We are going to have some regional grids, so there may need to be a diversity of approaches.
Remember that not everyone has dedicated funding for this: so the more we add requirements the lower the participation. I'd like to see us perhaps focus on getting notebooks/routines/code to read data in heterogenous formats into xarray, and let the user sort it out a bit.
I like the standard for catalogs, but we may also need to be flexible on zarr v. netCDF, and that may also depend on data center.
Hope that helps.
Andrew

Moin @andrewgettelman ,
Thanks a lot for your valuable questions and suggestions!
I've added draft to the title. Very good point. Most of the specifications, including level 9 and pressure levels stems from @bjorn-stevens . Maybe he can say more.
Pressure level output would (a) reduce the output quite a bit, and (b) ease comparisons, as people don't have to deal with the peculiarities of the different grids.
For compression, we have had very good experiences with zstd and lz4. A key point in this being well-formed compression chunks. Our initial attempts with compressing full 2D fields in one chunk with deflate (or CMIP6 whole 3D fields) proved bad, but in the nextGEMS Cycle 3 and 4, we used zstd and lz4 with up to 48 horizontal chunks and 3(?) chunks in the vertical, and that seems to work very well. And compression cuts the size of the data in half at no cost in quality. If a bit of rounding was applied before compression, way bigger factors could be reached (and how many digits do we really need on temperature or wind speed?).
For non-global grids, the situation is - hmm. Basically the healpix grid by definition is global, so what you'd do is to store a whole lot of NaN. On the disk, this is not really a problem, as the compressor eats long sequences of NaN for breakfast. In Memory it can be an issue, if you don't subset the data before loading (cdo would be an example for a rather poorly suited tool).
We might want to end with healpix for global, and "bring your own grid" for regional models, as those are very different anyways, but I think that's for a science steering committee to decide. I've stopped analyzing data years ago, so I'd leave these decisions to the people who actually want to work with the data.
I see the point that not every team has dedicated funding, but I've also seen the two previous DYAMOND initiatives, and that's a place I never want to visit again.
DYAMOND Summer is a complete mess on our disks and tapes with a zoo of file formats, variable names, etc., that makes processing and comparing cumbersome. It also sometimes requires fetching a whole dataset from the tapes to read one variable, as data was provided with all variables for one time step in a file.
For DYAMOND Winter, we spent several person months on trying to standardize the zoo of data that was dumped on our disks - all with mixed results, as we had to more-or-less guess, what's the closest match for the variable at hand, and file formats were anywhere between useful and a proof of human creativity.
On the other hand, in nextGEMS, we have made steps towards consolidation (not really getting there, but hey), and things got a lot better.
If the teams can provide examples of how to convert their data into healpix on pressure levels for comparisons, they can also apply that to their datasets, and do this exercise once instead of hundreds of users doing this one-by-one over the next years (we still have people asking for and analyzing DYAMOND Summer data from about 2018).
The intake catalogs actually are quite easy to create, and make people's lives a lot easier. They also work with netCDF files, so they don't really limit people too much in the ways they store the data.

Thanks @florianziemen for the reply. We do need to have some discussion of this at the infrastructure or the planning level.
Your comments and experience on trying to harmonize data are very valuable. We want to balance ease of use with ease of providing data. How much we force everything to be in the same format is going to be an interesting line to walk.
I think pushing catalogs is great, since they are easy to make and flexible.
Given your comments about doing compression wrong: I think we ought to back off it if we can. It will take some work it seems for each data set to figure this out (maybe it's easier with HealPix and it can be done after conversion to HealPix).
And as far as regional grids go: we probably do need to have some requirements on providing code to read data into xarray, and we might also want to require some sort of name mapping so that if files are NOT processed to have standard names there is at least a simple way to deal with it. Again, we can discuss this.
We might also think about some tests of data sets we could do in the fall (one 'Standard', one other global, maybe one regional').
Happy to discuss further.

I think we should merge this rather soon-ish to have a better basis for the discussions in the open.
We definitely still need to decide on things like mean vs instantaneous output.
I added a sentence saying that regional models might have a different grid need.


KubernetesCatalogWatch is missing in spring-cloud-kubernetes-client-discovery

Describe the bug
spring-cloud-starter-kubernetes  version 1.1.8.RELEASE
KubernetesCatalogWatch was exist is spring-cloud-kubernetes-discovery module.
but after update to 2.0.1
KubernetesCatalogWatch  is exists in spring-cloud-kubernetes-fabric8-discovery  module but it is missing in  spring-cloud-kubernetes-client-discovery module

You are right, looks like we are missing this feature for parity.

@ryanjbaxter duplicate of this. Can we close this one? My next todo is to fix that issue. thank you


Support for nvidia on Ubuntu Core 24

Hi.
I would like to add support for nvidia on Ubuntu Core 24 systems.  This means a couple of important changes:

Switching to base core24
Dropping support for nvidia on Ubuntu Core 22

Because of binary compatibility, it's not possible to support libraries from the host with different bases in a single snap.
I see two potential ways forward, but you may have other suggestions.

Create a new track for core22, and publish a final release
Create a new tracks for core22 and core24, with git branches for each, and try and maintain support for both.

I presume that sooner or later it would be desirable to update the base, even without this feature.
I've submitted an initial PR, just to get the ball rolling and start discussion.  Happy to debase as required once we find the best way forward and finish development.
I look forward to your comments, and any assistance you can provide.
Thank you!

Also just worth mentioning that the nvidia support in Ubuntu Core 24 is not quite ready for stable, but is expected to be in stable channel quite early next year.


travis publishes docker

Coverage remained the same at 74.226% when pulling eaef44a0418ed15c11f1600f44386ea79430f8ec on thiagobustamante:master into 91e0c7b46ea696d5f4df9dbd8f5f91f49bf363a8 on Leanty:master.


gn: Add the xwalk_core_library_docs target.

Unfortunately we still need to manually list all the files we want to
process as per
https://lists.crosswalk-project.org/pipermail/crosswalk-dev/2016-September/003927.html
We cannot call javadoc directly in GN, so a small wrapper,
generate_javadoc.py, has been added. It also knows how to deal with
.srcjar files, which is how the reflection wrapper code is currently
packaged.
The gyp version of the target has also been changed to use the wrapper
so that the targets in both build systems do not differ too much.
There is no xwalk_core_library_java_app_part target in GN, so we use
xwalk_core_java instead (and adjust gyp accordingly): the app_part
target was just merging xwalk_core_java.jar and the JARs it depends
on, but the documentation only needs the classes from
xwalk_core_java.jar itself, so there is no difference in the output.

@lincsoon @heke123


N2O SIO-16/SIO-05 scale conversion needs updating

Currently, it's just 1 as a placeholder.

Confirmed with Paul Krummel that the scale conversion from SIO-98 to SIO-16 is 1. NOTE: there was no SIO-05 scale for N2O.


Update docs to use py3k

I've updated print statements and a StringIO import to py3k. I may have missed other outdated pieces.

Coverage remained the same at 57.182% when pulling c086449818d53ed1f6e0eb3d6ef96946de2e6282 on clayadavis:docs_py3k into 0702b4bce19b7ec4d40f0c5db00ef0b6fa6ee6ce on ryanmcgrath:master.

Thanks!


iSpindel not logging data in Fermentrack for several days

Hi,
I am having an issue where my iSpindel's start logging data to Fermentrack and then drop off for up to four days before logging data again. Upon inspection when dry hopping the status light on the iSpindel is flashing. Could this be an  issue where Fermentrack is not 'awake' for that small window of time when the iSpindel transits data?

Thanks

I dont think this is an issue on fermentrack. It looks more like a network problem. Are you fermenting in a steel fermenter? Have you checked the network coverage from where you ferment?

@mp-se submitted a PR which adds a new field to the iSpindel page showing the timestamp data was last received from an iSpindel. This PR will be merged into dev after #462 (the Django 3.0 upgrade) goes live into master (so ~Friday, June 5th)

This is now merged into dev - I'll likely do another push to master in about ~1 month which will contain this as well


[Public Catalog] I/O Tessen event changes

Change event types to be consistent in naming convention. We currently have 2 events following the wrong pattern
Acceptance Criteria

[x] Change TDEV_InstantObservability_QuickstartViewToggle to TDEV_QuickstartViewToggle_InstantObservability
[x] Change TDEV_InstantObservability_QuickstartsCatalog to TDEV_QuickstartsCatalog_InstantObservability
[ ] Given we are only using 1 event type for the install button called TDEV_QuickstartInstall_instantObservability
[ ] We want to  add a field to the event to pass in the name of the button that was clicked, either install this quickstart OR see installation docs
[ ] Update WIKI with these changes

we found that TDEV_QuickstartCatalogSearch_instantObservability is actually being used to track catalog searches, so I updated the wiki
we also didn't instruement the list view / grid view toggle as I recall we deemed as not worthy of tracking


[6400] bulk placements create records pending import records

Context

We need to create placement rows for each csv row uploaded.
multiple rows are created for a single csv row
this is determined by the number of placement urns in a single row
these rows will later be used to create the actual placement records between a trainee and a school

Changes proposed in this pull request

adds service for creating placement rows after uploading a csv

Guidance to review

download the template after logging in as Annie Bell persona
add some URNs
upload and check the bulk_update_placement_rows table for new entries

 Bulk placements create records 1: parse and store upload import records


repository name

The crate is named simd-json (since we can't have simdjson ...) so would it make sense to rename the repository to reflect the crate name?
It will have very little (no?) outside impact but I'd love to get some feedback on this before going ahead.

This makes sense to me!


Inline Javascript Actions not Saved

Describe the Bug
I created a new workflow with an HTTP Request Trigger and then a Compose Shape.  I added an Inline Code Action after the Compose.  When you select the Execute JavaScript, it allows you to specify Actions that can be included within the JavaScript.  I added the Compose, but the Action Items are never saved. They are missing and do not appear in the Designer or in the Code.
I reviewed an existing workflow and it has the Action saved in the Code, but the new Designer does not include the Actions Item.
Steps to Reproduce the Bug or Issue

Create a new Workflow
Add an HTTP Request Trigger
Add a Compose Shape and select the HTTP Body as input
Add and Inline Code Shape
Select Execute Javascript and then specify or enter an Action Item
Select another shape, such as Compose
Return to the Inline code and the Action Item has disappeared
Saving the workflow does not save the Action Item either

Workflow JSON
{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Compose": {
                "inputs": "@triggerBody()",
                "runAfter": {},
                "type": "Compose"
            },
            "Execute_JavaScript_Code": {
                "inputs": {
                    "code": "return 0;"
                },
                "runAfter": {
                    "Compose": [
                        "SUCCEEDED"
                    ]
                },
                "type": "JavaScriptCode"
            }
        },
        "contentVersion": "<IP_ADDRESS>",
        "outputs": {},
        "triggers": {
            "When_a_HTTP_request_is_received": {
                "kind": "Http",
                "type": "Request"
            }
        }
    },
    "kind": "Stateful"
}

Expected behavior
As a user, I expected the Actions Item to be saved in the Execute Javascript Code Action when I selected another Action in the Designer
Screenshots or Videos
No response
Platform And Browser

OS: macOS
Browser: Microsoft Edge
Version: 107.0.1418.28

Additional context
No response
AB#16189545

One additional note.  I have an existing Inline Code from the regular Designer.  When I bring it up in the new designer, it shows a bunch of boxes and when you hover, it states:  The character U+00a0 is invisible.  Code view is fine, but on the Parameters tab it shows this quite a bit.

This is happening because our Array Editor integration is still a work-in-progress. On the bright side, this will be fixed in my next PR with the support of Complex Object Array Items


Update sbt-scalafix to 0.9.28

Updates ch.epfl.scala:sbt-scalafix from 0.9.27 to 0.9.28.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "ch.epfl.scala", artifactId = "sbt-scalafix" } ]

labels: sbt-plugin-update, semver-patch

Codecov Report

Merging #978 (475d90e) into develop (f8e8bc5) will increase coverage by 0.13%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #978      +/-   ##
===========================================
+ Coverage    89.54%   89.67%   +0.13%
===========================================
  Files          119      119
  Lines         9871     9871
  Branches       511      511
===========================================
+ Hits          8839     8852      +13
+ Misses        1032     1019      -13

Impacted Files
Coverage Δ

...scala/com/twitter/algebird/PredecessibleLaws.scala
71.42% <0.00%> (-14.29%)
:arrow_down:

.../main/scala/com/twitter/algebird/BloomFilter.scala
93.80% <0.00%> (ø)

.../scala/com/twitter/algebird/immutable/BitSet.scala
97.43% <0.00%> (+0.64%)
:arrow_up:

.../main/scala/com/twitter/algebird/Applicative.scala
55.17% <0.00%> (+3.44%)
:arrow_up:

...src/main/scala/com/twitter/algebird/Interval.scala
82.60% <0.00%> (+7.82%)
:arrow_up:

.../main/scala/com/twitter/algebird/Successible.scala
91.66% <0.00%> (+8.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8e8bc5...475d90e. Read the comment docs.


Cannot connect to Docker from WSL2 until you go to resourcer/wsl integration

File: docker-for-windows/wsl-tech-preview.md
With the new release of Docker Desktop integration with WSL2, the instructions in this page are not enough to connect with WSL - I had to go into Settings/Resources/WSL Integration and select my distro before this started working

closing issue as this has now been resolved


import problem with StringIO

I'm having this issue:
"....../lib/python3.6/background_task/models.py", line 8, in
from StringIO import StringIO
ModuleNotFoundError: No module named 'StringIO' "
And I don't really understand, because there's no such line in the models.py on this Git. But I found out there is in the old package django-background-task. I think the command  "pip install django-backround-tasks" is installing the wrong package and I don't know how to fix it. Please help!

You need to install 'django-background-tasks' instead of 'django-background-task'. That's the sublety.

That's the thing. I AM installing django-background-taskS.

That’s really strange. Do you use PyCharm or something like this ?

Nope

Can you tell me the version of your current package ?

In one of my tries I specifically installed the latest: "pip install django-background-tasks==1.2.0". Same issue. Pretty sure the other attempts also installed 1.2.0, if I recall correctly.

When you installed the package, the package django-compat is also here ?

Not sure. Will check that out.

@JTiemi make sure django-background-task is uninstalled as well.

@anthony-jou yes, the django-compat is there. And @philippeowagner, the django-background-task is not installed

What compat/background-tasks version did you installed?
Could you try from compat import StringIO on the command line?

Please reopen if your error still persists.


Code for Alignment

Hi,
Thanks for sharing the repo of the dataset. I wanted to compare the alignment method used in the paper based on Stable Matching for my work. Will it be possible for you to share the code?
Thanks!

Hi @saxenarohit, unfortunately we no longer have have access to the stabled matched alignments we used in this paper. We would suggest taking a look at
https://github.com/salesforce/booksum (Kryściński et al 2021): these authors collected a larger dataset, and also used our stable matching method. I think they provided the alignment scripts in that repo.
https://github.com/fladhak/creative-summ-data (Agrawal et al 2022): the Creative Summarization workshop was jointly organized by our team, the team for BookSum, and others. The datasets here are more up-to-date, and you could try running the alignment scripts from BookSum on this one.
Tagging my co-author @fladhak in case he has any other information.

Thanks @manestay for the pointers.
I can see booksum uses Matching package (https://github.com/daffidwilde/matching) for stable matching. This should work fine. Thanks for your help.

Glad I could help!


Inconsistent tagging in OpenAPI specifications

The Microsoft.ContainerService spec has inconsistent tagging.

Case 1: managedClusters
Case 2: ManagedClusters

For the documentation process to work for REST APIs, and grouped by tags, these values need to be consistent, as our ingestion pipeline is case-sensitive.
NOTE: We are looking at making our parser case-insensitive as well.

@sarangan12 do you happen to have any updates on this item?