It doesn't seem to work with Google Sheets

Thank you for the awesome work, made out with other host providers but I want a particular feature, being able to use CSV or TSV data from the public sharing link of google sheets... I have tried with getting the links and the underlied one as well (the link redirection) but it doesn't seem to work. Do I'm doing somethign wrong? And thank you again. 👍

Hi there,
first of all, thank you for using my script 👍
As for your specific request, are you experiencing issue with the content replacement feature or with those files not being proxied as a whole?
As for the content replacement feature, CORSflare will only perform it on text/html files, unless explicitly instructed otherwise, so maybe you just need to add the CSV and TSV content types to the CORSflare's replacement_content_types configuration constant (see here):
const replacement_content_types = ['text/html', 'text/csv', 'text/tsv', 'text/tab-separated-values'];

If the issue is about those files not being proxied, please send me your CORSflare.js file so that I can test it out and see what's going on: be sure to obfuscate your content IDs and/or tokens, just leave the root URL so that I can understand how to properly reproduce it with my own Google Sheet files.

The question is specific about the content not being proxified, I'm not using other features and I don't mind anyone seeing the data, it's just mock-up. 😃
Here is the code, I forgot to mention that I was using papaparse to parse the csv 😅, but I tested if it worked on other platforms just to be sure, it was flawless on dropbox and netlify, so, I guessed it was Google problem, the code:
<html>
<head>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/PapaParse/5.3.0/papaparse.min.js"></script>
</head>
<body>
  <script>
    Papa.parse("https://square-lake-54f9.gomerk.workers.dev/spreadsheets/d/e/2PACX-1vSe-UCupCM9CIkK-CP01nOT7yyW5kC0mAGnqcDoEgaiWgKNsSytZS4js2flxi_ATogt8dthIpV46NiT/pub?gid=1814306385&single=true&output=csv", {
      download: true,
      complete: function(results) {
      console.log(results);
    }
  });
  </script>
</body>
</html>

And the CORSflare.js code:
// ----------------------------------------------------------------------------------
// CORSflare - v1.0.4
// ref.: https://github.com/Darkseal/CORSflare
// A lightweight JavaScript CORS Reverse Proxy designed to run in a Cloudflare Worker
// ----------------------------------------------------------------------------------

// ----------------------------------------------------------------------------------
// CONFIGURATION SETTINGS
// ----------------------------------------------------------------------------------

// The hostname of the upstream website to proxy(example: `www.google.com`).
const upstream = 'docs.google.com';

// ** This is the public gsheet - csv ** //
//https://docs.google.com/spreadsheets/d/e/2PACX-1vSe-UCupCM9CIkK-CP01nOT7yyW5kC0mAGnqcDoEgaiWgKNsSytZS4js2flxi_ATogt8dthIpV46NiT/pub?gid=1814306385&single=true&output=csv

// The hostname of the upstream website to proxy for requests coming from mobile devices(example: `www.google.com`).
// if the upstream website doesn't have a dedicated hostname for mobile devices, you can set it to NULL.
const upstream_mobile = null;

// Custom pathname for the upstream website ('/' will work for most scenarios)
const upstream_path = '/';

// An array of countries and regions that won't be able to use the proxy.
const blocked_regions = ['CN', 'KP', 'SY', 'PK', 'CU'];

// An array of IP addresses that won't be able to use the proxy.
const blocked_ip_addresses = ['<IP_ADDRESS>', '<IP_ADDRESS>'];

// Set this value to TRUE to fetch the upstream website using HTTPS, FALSE to use HTTP.
// If the upstream website doesn't support HTTPS, this must be set to FALSE; also, if the proxy is HTTPS,
// you'll need to enable the replacement_rules rule to HTTPS proxy an HTTP-only website (see below).
const https = true;

// Set this value to TRUE to forcefully apply the "SameSite=None" and "Secure" directives to all cookies generated by the upstream.
// If you plan to put this proxy within a <iframe> and allow users to POST FORM data, you might need this option to make auth cookies work.
// ref.: https://sites.google.com/a/chromium.org/dev/updates/same-site/faq
// ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite
// NOTE: This is an experimental feature and could prevent some cookies from being generated due to a known bug of the Fetch API
// affecting the "Set-Cookie" response headers: use it at your own risk.
const set_cookie_samesite_none = false;

// an array of HTTP Response Headers to add (or to update, in case they're already present in the upstream response)
const http_response_headers_set = {
    // use these headers to bypass DENY and SAMEORIGIN policies for IFRAME, OBJECT, EMBED and so on for most browsers.
    // NOTE: be sure to replace "https://www.example.com" with the domain of the HTML page containing the IFRAME.
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
    'X-Frame-Options': 'ALLOW FROM https://www.example.com', // IE
    'Content-Security-Policy': "frame-ancestors 'self' https://www.example.com;", // Chrome, Firefox, etc.

    // use this header to bypass the same-origin policy for XMLHttpRequest, Fetch API and so on
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin
    'Access-Control-Allow-Origin': '*',

    // use this header to accept (and respond to) preflight requests when the request's credentials mode is set to 'include'
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials
    'Access-Control-Allow-Credentials': true,

    // use this header to override the Cache-Control settings of the upstream pages. Allowed values include:
    // 'must-revalidate', 'no-cache', 'no-store', 'no-transform', 'public', 'private',
    // 'proxy-revalidate', 'max-age=<seconds>', 's-maxage=<seconds>'.
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
    // 'Cache-Control': 'no-cache'
};

// an array of HTTP Response Headers to delete (if present in the upstream response)
const http_response_headers_delete = [
    'Content-Security-Policy-Report-Only',
    'Clear-Site-Data'
];

// ----------------------------------------------------------------------------------
// TEXT REPLACEMENT RULES
// ----------------------------------------------------------------------------------
// The replacement_rules array can be used to configure the text replacement rules
// that will be applied by the proxy before serving any text/html resource back to the user.
// The common usage of such rules is to "fix" non-standard internal URLs and/or local paths
// within the upstream's HTML pages (css, js, internal links, custom fonts, and so on) and force them
// to pass to the proxy; however, they can also be used to alter the response content in various ways
// (change a logo, modify the page title, add a custom css/js, and so on).

// Each rule must be defined in the following way:

// '<source_string>' : '<replacement_string>'

// The following dynamic placeholder can be used within the source and replacement strings:

// {upstream_hostname}  : will be replaced with the upstream's hostname
// {proxy_hostname}     : will be replaced with this proxy's hostname

// HINT: Rules are processed from top to bottom: put the most specific rules before the generic ones.

const replacement_rules = {

    // enable this rule only if you need to HTTPS proxy an HTTP-only website
    'http://{upstream_hostname}/': 'https://{proxy_hostname}/',

    // this rule should be always enabled (replaces the upstream hostname for internal links, CSS, JS, and so on)
    '{upstream_hostname}': '{proxy_hostname}',

}

// the replacement_rules will be only applied to the returned content
// with the following content types specified by the replacement_content_types array.
const replacement_content_types = ['text/csv, text/html'];

// Set this value to TRUE to allow RegEx syntax in replacement rules (see URLs below for details):
// - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions
// - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Cheatsheet
// NOTE: if RegEx syntax is enabled, RegEx special chars in search patterns will have to be escaped
// using a double back slash (\\) accordingly.
const replacement_use_regex = true;

// ----------------------------------------------------------------------------------
// MAIN CODE
// ----------------------------------------------------------------------------------

var regexp_upstreamHostname = (replacement_use_regex)
    ? new RegExp('{upstream_hostname}', 'g')
    : null;
var regexp_proxyHostname = (replacement_use_regex)
    ? new RegExp('{proxy_hostname}', 'g')
    : null;

addEventListener('fetch', event => {
    event.respondWith(fetchAndApply(event.request));
})

async function fetchAndApply(request) {
    var r = request.headers.get('cf-ipcountry');
    const region = (r) ? r.toUpperCase() : null;
    const ip_address = request.headers.get('cf-connecting-ip');
    const user_agent = request.headers.get('user-agent');

    let response = null;
    let url = new URL(request.url);
    let url_hostname = url.hostname;

    if (https == true) {
        url.protocol = 'https:';
    } else {
        url.protocol = 'http:';
    }

    if (upstream_mobile && await is_mobile_user_agent(user_agent)) {
        var upstream_domain = upstream_mobile;
    } else {
        var upstream_domain = upstream;
    }

    url.host = upstream_domain;
    if (url.pathname == '/') {
        url.pathname = upstream_path;
    } else {
        url.pathname = upstream_path + url.pathname;
    }

    if (blocked_regions.includes(region) || blocked_ip_addresses.includes(ip_address)) {
        response = new Response('Access denied', {
            status: 403
        });
    } else {
        let method = request.method;
        let request_headers = request.headers;
        let new_request_headers = new Headers(request_headers);
        let request_content_type = new_request_headers.get('content-type');

        new_request_headers.set('Host', upstream_domain);
        new_request_headers.set('Origin', upstream_domain);
        new_request_headers.set('Referer', url.protocol + '//' + url_hostname);

        var params = {
            method: method,
            headers: new_request_headers,
            // this is required to properly handle standard HTTP redirects, as we need to alter the "location" header:
            // the default "follow" value would auto-resolve them with the upstream URL, which is not what we want.
            redirect: 'manual'
        }

        // if the request is supposed to contain Form Data, populates the request body accordingly
        if (method.toUpperCase() === "POST" && request_content_type) {
            let request_content_type_toLower = request_content_type.toLowerCase();
            if (request_content_type_toLower.includes("application/x-www-form-urlencoded")
                || request_content_type_toLower.includes("multipart/form-data")
                || request_content_type_toLower.includes("application/json")
            ) {
                let reqText = await request.text(); // TODO: this won't work for multipart/form-data
                if (reqText) {
                    params.body = reqText;
                }
            }
        }

        let original_response = await fetch(url.href, params);

        connection_upgrade = new_request_headers.get("Upgrade");
        if (connection_upgrade && connection_upgrade.toLowerCase() == "websocket") {
            return original_response;
        }

        let original_response_clone = original_response.clone();
        let response_headers = original_response_clone.headers;
        let response_status = original_response_clone.status;
        let original_text = null;
        let new_response_headers = new Headers(response_headers);
        let new_response_status = response_status;

        if (http_response_headers_set) {
            for (let k in http_response_headers_set) {
                var v = http_response_headers_set[k];
                new_response_headers.set(k, v);
            }
        }

        if (http_response_headers_delete) {
            for (let k of http_response_headers_delete) {
                new_response_headers.delete(k);
            }
        }

        // Patch "x-pjax-url" header to handle pushState ajax redirects
        if (new_response_headers.get("x-pjax-url")) {
            new_response_headers.set("x-pjax-url", new_response_headers.get("x-pjax-url")
                .replace(url.protocol + "//", "https://")
                .replace(upstream_domain, url_hostname));
        }

        // Patch "location" header to handle standard 301/302/303/307/308 HTTP redirects
        if (new_response_headers.get("location")) {
            new_response_headers.set("location", new_response_headers.get("location")
                .replace(url.protocol + "//", "https://")
                .replace(upstream_domain, url_hostname));
        }

        // Patch "set-cookie" headers by forcefully apply "SameSite=None" and "Secure" directives to allow cross-domain usage
        // (if "set_cookie_samesite_none" is set to TRUE: see that configuration option's comment block for details and references)
        if (set_cookie_samesite_none && new_response_headers.has("set-cookie")) {
            // NOTE: unfortunately the Fetch API Headers object doesn't support multiple Set-Cookie headers due to a bug in Fetch API's
            // "Headers" interface, as they are merged into a single comma-separated string (which is incompatible with most browsers).
            // ref.: https://stackoverflow.com/questions/63204093/how-to-get-set-multiple-set-cookie-response-headers-using-fetch-api
            // For that very reason, we can only support the * first * set-cookie header here.
            var firstCookie = new_response_headers.get("set-cookie").split(',').shift();
            new_response_headers.set("set-cookie", firstCookie
                .split("SameSite=Lax; Secure").join("")
                .split("SameSite=Lax").join("")
                .split("SameSite=Strict; Secure").join("")
                .split("SameSite=Strict").join("")
                .split("SameSite=None; Secure").join("")
                .split("SameSite=None").join("")
                .replace(/^;+$/g, '')
                + "; SameSite=None; Secure");
        }

        let response_content_type = new_response_headers.get('content-type');
        if (response_content_type
            && replacement_content_types.some(v => response_content_type.toLowerCase().includes(v))) {
            original_text = await replace_response_text(original_response_clone, upstream_domain, url_hostname);
        } else {
            original_text = original_response_clone.body;
        }

        response = new Response(original_text, {
            status: new_response_status,
            headers: new_response_headers
        })
    }
    return response;
}

async function replace_response_text(response, upstream_domain, host_name) {
    let text = await response.text()
    if (replacement_rules) {
        for (let k in replacement_rules) {
            var v = replacement_rules[k];

            if (replacement_use_regex) {
                k = k.replace(regexp_upstreamHostname, upstream_domain);
                k = k.replace(regexp_proxyHostname, host_name);
                v = v.replace(regexp_upstreamHostname, upstream_domain);
                v = v.replace(regexp_proxyHostname, host_name);
                text = text.replace(new RegExp(k, 'g'), v);
            }
            else {
                k = k.split('{upstream_hostname}').join(upstream_domain);
                k = k.split('{proxy_hostname}').join(host_name);
                v = v.split('{upstream_hostname}').join(upstream_domain);
                v = v.split('{proxy_hostname}').join(host_name);
                text = text.split(k).join(v);
            }
        }
    }
    return text;
}

async function is_mobile_user_agent(user_agent_info) {
    var agents = ["Android", "iPhone", "SymbianOS", "Windows Phone", "iPad", "iPod"];
    for (var v = 0; v < agents.length; v++) {
        if (user_agent_info.indexOf(agents[v]) > 0) {
            return true;
        }
    }
    return false;
}

Hello there,
The issue with Google Sheets is due to the fact that the URL you're calling doesn't directly serve the CSV file: instead, it issues a 307 redirect to another page (located on a VARIABLE EXTERNAL domain, such as doc-0s-94-sheets.googleusercontent.com) which not only also has a CORS policy applied, but does only accept a single request before expiring due to a OTP unique string placed in the URL itself: to put it in other words, it's a request/redirect nightmare explicitly designed to prevent exploits such as the one I'm using with CORSflare.
That said, I still managed to workaround it with a little tweak of the CORSflare source code: in a nutshell, the proxy now detects the redirects to "googleusercontent.com" URLs and - if that's the case - sends the redirect back to the client adding a dedicated GET parameter that - if present - will be used to setup a new upstream. I'm not sure if the explanation is clear enough, but I tested it and it works :)
I'll see if I'll be able to add this tweak as a "new feature" in an upcoming version of CORSflare.
// ----------------------------------------------------------------------------------
// CORSflare - v1.0.4
// ref.: https://github.com/Darkseal/CORSflare
// A lightweight JavaScript CORS Reverse Proxy designed to run in a Cloudflare Worker
// ----------------------------------------------------------------------------------

// ----------------------------------------------------------------------------------
// CONFIGURATION SETTINGS
// ----------------------------------------------------------------------------------

// The hostname of the upstream website to proxy(example: `www.google.com`).
const upstream = 'docs.google.com';

// ** This is the public gsheet - csv ** //
// https://docs.google.com/spreadsheets/d/e/2PACX-1vSe-UCupCM9CIkK-CP01nOT7yyW5kC0mAGnqcDoEgaiWgKNsSytZS4js2flxi_ATogt8dthIpV46NiT/pub?gid=1814306385&single=true&output=csv

// The hostname of the upstream website to proxy for requests coming from mobile devices(example: `www.google.com`).
// if the upstream website doesn't have a dedicated hostname for mobile devices, you can set it to NULL.
const upstream_mobile = null;

// Custom pathname for the upstream website ('/' will work for most scenarios)
const upstream_path = '/';

// An array of countries and regions that won't be able to use the proxy.
const blocked_regions = ['CN', 'KP', 'SY', 'PK', 'CU'];

// An array of IP addresses that won't be able to use the proxy.
const blocked_ip_addresses = ['<IP_ADDRESS>', '<IP_ADDRESS>'];

// Set this value to TRUE to fetch the upstream website using HTTPS, FALSE to use HTTP.
// If the upstream website doesn't support HTTPS, this must be set to FALSE; also, if the proxy is HTTPS,
// you'll need to enable the replacement_rules rule to HTTPS proxy an HTTP-only website (see below).
const https = true;

// Set this value to TRUE to forcefully apply the "SameSite=None" and "Secure" directives to all cookies generated by the upstream.
// If you plan to put this proxy within a <iframe> and allow users to POST FORM data, you might need this option to make auth cookies work.
// ref.: https://sites.google.com/a/chromium.org/dev/updates/same-site/faq
// ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite
// NOTE: This is an experimental feature and could prevent some cookies from being generated due to a known bug of the Fetch API
// affecting the "Set-Cookie" response headers: use it at your own risk.
const set_cookie_samesite_none = false;

// an array of HTTP Response Headers to add (or to update, in case they're already present in the upstream response)
const http_response_headers_set = {
    // use these headers to bypass DENY and SAMEORIGIN policies for IFRAME, OBJECT, EMBED and so on for most browsers.
    // NOTE: be sure to replace "https://www.example.com" with the domain of the HTML page containing the IFRAME.
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
    'X-Frame-Options': 'ALLOW FROM https://www.example.com', // IE
    'Content-Security-Policy': "frame-ancestors 'self' https://www.example.com;", // Chrome, Firefox, etc.

    // use this header to bypass the same-origin policy for XMLHttpRequest, Fetch API and so on
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin
    'Access-Control-Allow-Origin': '*',

    // use this header to accept (and respond to) preflight requests when the request's credentials mode is set to 'include'
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials
    'Access-Control-Allow-Credentials': true,

    // use this header to override the Cache-Control settings of the upstream pages. Allowed values include:
    // 'must-revalidate', 'no-cache', 'no-store', 'no-transform', 'public', 'private',
    // 'proxy-revalidate', 'max-age=<seconds>', 's-maxage=<seconds>'.
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
    // 'Cache-Control': 'no-cache'
};

// an array of HTTP Response Headers to delete (if present in the upstream response)
const http_response_headers_delete = [
    'Content-Security-Policy-Report-Only',
    'Clear-Site-Data'
];

// ----------------------------------------------------------------------------------
// TEXT REPLACEMENT RULES
// ----------------------------------------------------------------------------------
// The replacement_rules array can be used to configure the text replacement rules
// that will be applied by the proxy before serving any text/html resource back to the user.
// The common usage of such rules is to "fix" non-standard internal URLs and/or local paths
// within the upstream's HTML pages (css, js, internal links, custom fonts, and so on) and force them
// to pass to the proxy; however, they can also be used to alter the response content in various ways
// (change a logo, modify the page title, add a custom css/js, and so on).

// Each rule must be defined in the following way:

// '<source_string>' : '<replacement_string>'

// The following dynamic placeholder can be used within the source and replacement strings:

// {upstream_hostname}  : will be replaced with the upstream's hostname
// {proxy_hostname}     : will be replaced with this proxy's hostname

// HINT: Rules are processed from top to bottom: put the most specific rules before the generic ones.

const replacement_rules = {

    // enable this rule only if you need to HTTPS proxy an HTTP-only website
    'http://{upstream_hostname}/': 'https://{proxy_hostname}/',

    // this rule should be always enabled (replaces the upstream hostname for internal links, CSS, JS, and so on)
    '{upstream_hostname}': '{proxy_hostname}',

}

// the replacement_rules will be only applied to the returned content
// with the following content types specified by the replacement_content_types array.
const replacement_content_types = ['text/csv, text/html'];

// Set this value to TRUE to allow RegEx syntax in replacement rules (see URLs below for details):
// - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions
// - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Cheatsheet
// NOTE: if RegEx syntax is enabled, RegEx special chars in search patterns will have to be escaped
// using a double back slash (\\) accordingly.
const replacement_use_regex = true;

// ----------------------------------------------------------------------------------
// MAIN CODE
// ----------------------------------------------------------------------------------

var regexp_upstreamHostname = (replacement_use_regex)
    ? new RegExp('{upstream_hostname}', 'g')
    : null;
var regexp_proxyHostname = (replacement_use_regex)
    ? new RegExp('{proxy_hostname}', 'g')
    : null;

addEventListener('fetch', event => {
    event.respondWith(fetchAndApply(event.request));
})

async function fetchAndApply(request) {
    var r = request.headers.get('cf-ipcountry');
    const region = (r) ? r.toUpperCase() : null;
    const ip_address = request.headers.get('cf-connecting-ip');
    const user_agent = request.headers.get('user-agent');

    let response = null;
    let url = new URL(request.url);
    let url_hostname = url.hostname;

    if (https == true) {
        url.protocol = 'https:';
    } else {
        url.protocol = 'http:';
    }

    var upstream_header = url.searchParams.get('CORSflare_upstream');

    if (upstream_header) {
        var upstream_domain = upstream_header;
    }
    else if (upstream_mobile && await is_mobile_user_agent(user_agent)) {
        var upstream_domain = upstream_mobile;
    } else {
        var upstream_domain = upstream;
    }

    url.host = upstream_domain;
    if (url.pathname == '/') {
        url.pathname = upstream_path;
    } else {
        url.pathname = upstream_path + url.pathname;
    }

    if (blocked_regions.includes(region) || blocked_ip_addresses.includes(ip_address)) {
        response = new Response('Access denied', {
            status: 403
        });
    } else {
        let method = request.method;
        let request_headers = request.headers;
        let new_request_headers = new Headers(request_headers);
        let request_content_type = new_request_headers.get('content-type');

        new_request_headers.set('Host', upstream_domain);
        new_request_headers.set('Origin', upstream_domain);
        new_request_headers.set('Referer', url.protocol + '//' + url_hostname);

        var params = {
            method: method,
            headers: new_request_headers,
            // this is required to properly handle standard HTTP redirects, as we need to alter the "location" header:
            // the default "follow" value would auto-resolve them with the upstream URL, which is not what we want.
            redirect: 'manual'
        }

        // if the request is supposed to contain Form Data, populates the request body accordingly
        if (method.toUpperCase() === "POST" && request_content_type) {
            let request_content_type_toLower = request_content_type.toLowerCase();
            if (request_content_type_toLower.includes("application/x-www-form-urlencoded")
                || request_content_type_toLower.includes("multipart/form-data")
                || request_content_type_toLower.includes("application/json")
            ) {
                let reqText = await request.text(); // TODO: this won't work for multipart/form-data
                if (reqText) {
                    params.body = reqText;
                }
            }
        }

        let original_response = await fetch(url.href, params);

        connection_upgrade = new_request_headers.get("Upgrade");
        if (connection_upgrade && connection_upgrade.toLowerCase() == "websocket") {
            return original_response;
        }

        let original_response_clone = original_response.clone();
        let response_headers = original_response_clone.headers;
        let response_status = original_response_clone.status;
        let original_text = null;
        let new_response_headers = new Headers(response_headers);
        let new_response_status = response_status;

        if (http_response_headers_set) {
            for (let k in http_response_headers_set) {
                var v = http_response_headers_set[k];
                new_response_headers.set(k, v);
            }
        }

        if (http_response_headers_delete) {
            for (let k of http_response_headers_delete) {
                new_response_headers.delete(k);
            }
        }

        // Patch "x-pjax-url" header to handle pushState ajax redirects
        if (new_response_headers.get("x-pjax-url")) {
            new_response_headers.set("x-pjax-url", new_response_headers.get("x-pjax-url")
                .replace(url.protocol + "//", "https://")
                .replace(upstream_domain, url_hostname));
        }

        // Patch "location" header to handle standard 301/302/303/307/308 HTTP redirects
        if (new_response_headers.get("location")) {

            var location = new_response_headers.get("location");
            if (location.includes("googleusercontent.com"))
            {
                var new_upstream = location.substring(8, location.indexOf("/",8));
                location = location + "&CORSflare_upstream=" + new_upstream;
                new_response_headers.set("location", location
                    .replace(url.protocol + "//", "https://")
                    .replace(new_upstream, url_hostname));
            }
            else {
            new_response_headers.set("location", new_response_headers.get("location")
                .replace(url.protocol + "//", "https://")
                .replace(upstream_domain, url_hostname));
            }
        }

        // Patch "set-cookie" headers by forcefully apply "SameSite=None" and "Secure" directives to allow cross-domain usage
        // (if "set_cookie_samesite_none" is set to TRUE: see that configuration option's comment block for details and references)
        if (set_cookie_samesite_none && new_response_headers.has("set-cookie")) {
            // NOTE: unfortunately the Fetch API Headers object doesn't support multiple Set-Cookie headers due to a bug in Fetch API's
            // "Headers" interface, as they are merged into a single comma-separated string (which is incompatible with most browsers).
            // ref.: https://stackoverflow.com/questions/63204093/how-to-get-set-multiple-set-cookie-response-headers-using-fetch-api
            // For that very reason, we can only support the * first * set-cookie header here.
            var firstCookie = new_response_headers.get("set-cookie").split(',').shift();
            new_response_headers.set("set-cookie", firstCookie
                .split("SameSite=Lax; Secure").join("")
                .split("SameSite=Lax").join("")
                .split("SameSite=Strict; Secure").join("")
                .split("SameSite=Strict").join("")
                .split("SameSite=None; Secure").join("")
                .split("SameSite=None").join("")
                .replace(/^;+$/g, '')
                + "; SameSite=None; Secure");
        }

        let response_content_type = new_response_headers.get('content-type');
        if (response_content_type
            && replacement_content_types.some(v => response_content_type.toLowerCase().includes(v))) {
            original_text = await replace_response_text(original_response_clone, upstream_domain, url_hostname);
        } else {
            original_text = original_response_clone.body;
        }

        response = new Response(original_text, {
            status: new_response_status,
            headers: new_response_headers
        })
    }
    return response;
}

async function replace_response_text(response, upstream_domain, host_name) {
    let text = await response.text()
    if (replacement_rules) {
        for (let k in replacement_rules) {
            var v = replacement_rules[k];

            if (replacement_use_regex) {
                k = k.replace(regexp_upstreamHostname, upstream_domain);
                k = k.replace(regexp_proxyHostname, host_name);
                v = v.replace(regexp_upstreamHostname, upstream_domain);
                v = v.replace(regexp_proxyHostname, host_name);
                text = text.replace(new RegExp(k, 'g'), v);
            }
            else {
                k = k.split('{upstream_hostname}').join(upstream_domain);
                k = k.split('{proxy_hostname}').join(host_name);
                v = v.split('{upstream_hostname}').join(upstream_domain);
                v = v.split('{proxy_hostname}').join(host_name);
                text = text.split(k).join(v);
            }
        }
    }
    return text;
}

async function is_mobile_user_agent(user_agent_info) {
    var agents = ["Android", "iPhone", "SymbianOS", "Windows Phone", "iPad", "iPod"];
    for (var v = 0; v < agents.length; v++) {
        if (user_agent_info.indexOf(agents[v]) > 0) {
            return true;
        }
    }
    return false;
}

Sadly, it doesn't seem to be working for me (though, I'm surely doing something wrong), if I understood right:

Download and use the lastest Corsflare
Change the "upstream_get_parameter" to true
Put "upstream" to docs.google.com
Use same html the I was using before.

Wields a 404 error:

I will post my Corsflare again:
// ----------------------------------------------------------------------------------
// CORSflare - v1.0.5
// ref.: https://github.com/Darkseal/CORSflare
// A lightweight JavaScript CORS Reverse Proxy designed to run in a Cloudflare Worker
// ----------------------------------------------------------------------------------

// ----------------------------------------------------------------------------------
// CONFIGURATION SETTINGS
// ----------------------------------------------------------------------------------

// The hostname of the upstream website to proxy(example: `www.google.com`).
const upstream = 'docs.google.com';

// The hostname of the upstream website to proxy for requests coming from mobile devices(example: `www.google.com`).
// if the upstream website doesn't have a dedicated hostname for mobile devices, you can set it to NULL.
const upstream_mobile = null;

// Custom pathname for the upstream website ('/' will work for most scenarios)
const upstream_path = '/';

// Set it to TRUE to allow the default upstream to be overridden with a customizable GET parameter (see `upstream_get_parameter` below), FALSE to prevent that.
// WARNING: this feature will allow third-parties to replace this proxy's default upstream with an upstream of their choice: activate it only if you know
// what you're doing (see ref. below): also, it could be wise to change the default `upstream_get_parameter` with a unique string to reduce the risk of hijacks.
// ref.: https://github.com/Darkseal/CORSflare/issues/1
const upstream_allow_override = true;

// The GET parameter that can be used to override the default upstream if `upstream_allow_override` is set to TRUE.
const upstream_get_parameter = 'CORSflare_upstream';

// An array of countries and regions that won't be able to use the proxy.
const blocked_regions = ['CN', 'KP', 'SY', 'PK', 'CU'];

// An array of IP addresses that won't be able to use the proxy.
const blocked_ip_addresses = ['<IP_ADDRESS>', '<IP_ADDRESS>'];

// Set this value to TRUE to fetch the upstream website using HTTPS, FALSE to use HTTP.
// If the upstream website doesn't support HTTPS, this must be set to FALSE; also, if the proxy is HTTPS,
// you'll need to enable the replacement_rules rule to HTTPS proxy an HTTP-only website (see below).
const https = true;

// Set this value to TRUE to forcefully apply the "SameSite=None" and "Secure" directives to all cookies generated by the upstream.
// If you plan to put this proxy within a <iframe> and allow users to POST FORM data, you might need this option to make auth cookies work.
// ref.: https://sites.google.com/a/chromium.org/dev/updates/same-site/faq
// ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite
// NOTE: This is an experimental feature and could prevent some cookies from being generated due to a known bug of the Fetch API
// affecting the "Set-Cookie" response headers: use it at your own risk.
const set_cookie_samesite_none = false;

// an array of HTTP Response Headers to add (or to update, in case they're already present in the upstream response)
const http_response_headers_set = {
    // use these headers to bypass DENY and SAMEORIGIN policies for IFRAME, OBJECT, EMBED and so on for most browsers.
    // NOTE: be sure to replace "https://www.example.com" with the domain of the HTML page containing the IFRAME.
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
    'X-Frame-Options': 'ALLOW FROM https://www.example.com', // IE
    'Content-Security-Policy': "frame-ancestors 'self' https://www.example.com;", // Chrome, Firefox, etc.

    // use this header to bypass the same-origin policy for XMLHttpRequest, Fetch API and so on
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin
    'Access-Control-Allow-Origin': '*',

    // use this header to accept (and respond to) preflight requests when the request's credentials mode is set to 'include'
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials
    'Access-Control-Allow-Credentials': true,

    // use this header to override the Cache-Control settings of the upstream pages. Allowed values include:
    // 'must-revalidate', 'no-cache', 'no-store', 'no-transform', 'public', 'private',
    // 'proxy-revalidate', 'max-age=<seconds>', 's-maxage=<seconds>'.
    // ref.: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
    // 'Cache-Control': 'no-cache'
};

// an array of HTTP Response Headers to delete (if present in the upstream response)
const http_response_headers_delete = [
    'Content-Security-Policy-Report-Only',
    'Clear-Site-Data'
];

// ----------------------------------------------------------------------------------
// TEXT REPLACEMENT RULES
// ----------------------------------------------------------------------------------
// The replacement_rules array can be used to configure the text replacement rules
// that will be applied by the proxy before serving any text/html resource back to the user.
// The common usage of such rules is to "fix" non-standard internal URLs and/or local paths
// within the upstream's HTML pages (css, js, internal links, custom fonts, and so on) and force them
// to pass to the proxy; however, they can also be used to alter the response content in various ways
// (change a logo, modify the page title, add a custom css/js, and so on).

// Each rule must be defined in the following way:

// '<source_string>' : '<replacement_string>'

// The following dynamic placeholder can be used within the source and replacement strings:

// {upstream_hostname}  : will be replaced with the upstream's hostname
// {proxy_hostname}     : will be replaced with this proxy's hostname

// HINT: Rules are processed from top to bottom: put the most specific rules before the generic ones.

const replacement_rules = {

    // enable this rule only if you need to HTTPS proxy an HTTP-only website
    'http://{upstream_hostname}/': 'https://{proxy_hostname}/',

    // this rule should be always enabled (replaces the upstream hostname for internal links, CSS, JS, and so on)
    '{upstream_hostname}': '{proxy_hostname}',

}

// the replacement_rules will be only applied to the returned content
// with the following content types specified by the replacement_content_types array.
const replacement_content_types = ['text/csv, text/html'];

// Set this value to TRUE to allow RegEx syntax in replacement rules (see URLs below for details):
// - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions
// - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Cheatsheet
// NOTE: if RegEx syntax is enabled, RegEx special chars in search patterns will have to be escaped
// using a double back slash (\\) accordingly.
const replacement_use_regex = true;

// ----------------------------------------------------------------------------------
// MAIN CODE
// ----------------------------------------------------------------------------------

var regexp_upstreamHostname = (replacement_use_regex)
    ? new RegExp('{upstream_hostname}', 'g')
    : null;
var regexp_proxyHostname = (replacement_use_regex)
    ? new RegExp('{proxy_hostname}', 'g')
    : null;

addEventListener('fetch', event => {
    event.respondWith(fetchAndApply(event.request));
})

async function fetchAndApply(request) {
    var r = request.headers.get('cf-ipcountry');
    const region = (r) ? r.toUpperCase() : null;
    const ip_address = request.headers.get('cf-connecting-ip');
    const user_agent = request.headers.get('user-agent');

    let response = null;
    let url = new URL(request.url);
    let url_hostname = url.hostname;
    let upstream_GET = (upstream_allow_override)
        ? url.searchParams.get(upstream_get_parameter)
        : null;

    if (https == true) {
        url.protocol = 'https:';
    } else {
        url.protocol = 'http:';
    }

    var upstream_domain = null;
    if (upstream_GET) {
        upstream_domain = upstream_GET;
    }
    if (upstream_mobile && await is_mobile_user_agent(user_agent)) {
        upstream_domain = upstream_mobile;
    } else {
        upstream_domain = upstream;
    }

    url.host = upstream_domain;
    if (url.pathname == '/') {
        url.pathname = upstream_path;
    } else {
        url.pathname = upstream_path + url.pathname;
    }

    if (blocked_regions.includes(region) || blocked_ip_addresses.includes(ip_address)) {
        response = new Response('Access denied', {
            status: 403
        });
    } else {
        let method = request.method;
        let request_headers = request.headers;
        let new_request_headers = new Headers(request_headers);
        let request_content_type = new_request_headers.get('content-type');

        new_request_headers.set('Host', upstream_domain);
        new_request_headers.set('Origin', upstream_domain);
        new_request_headers.set('Referer', url.protocol + '//' + url_hostname);

        var params = {
            method: method,
            headers: new_request_headers,
            // this is required to properly handle standard HTTP redirects, as we need to alter the "location" header:
            // the default "follow" value would auto-resolve them with the upstream URL, which is not what we want.
            redirect: 'manual'
        }

        // if the request is supposed to contain Form Data, populates the request body accordingly
        if (method.toUpperCase() === "POST" && request_content_type) {
            let request_content_type_toLower = request_content_type.toLowerCase();
            if (request_content_type_toLower.includes("application/x-www-form-urlencoded")
                || request_content_type_toLower.includes("multipart/form-data")
                || request_content_type_toLower.includes("application/json")
            ) {
                let reqText = await request.text(); // TODO: this won't work for multipart/form-data
                if (reqText) {
                    params.body = reqText;
                }
            }
        }

        let original_response = await fetch(url.href, params);

        connection_upgrade = new_request_headers.get("Upgrade");
        if (connection_upgrade && connection_upgrade.toLowerCase() == "websocket") {
            return original_response;
        }

        let original_response_clone = original_response.clone();
        let response_headers = original_response_clone.headers;
        let response_status = original_response_clone.status;
        let original_text = null;
        let new_response_headers = new Headers(response_headers);
        let new_response_status = response_status;

        if (http_response_headers_set) {
            for (let k in http_response_headers_set) {
                var v = http_response_headers_set[k];
                new_response_headers.set(k, v);
            }
        }

        if (http_response_headers_delete) {
            for (let k of http_response_headers_delete) {
                new_response_headers.delete(k);
            }
        }

        // Patch "x-pjax-url" header to handle pushState ajax redirects
        if (new_response_headers.get("x-pjax-url")) {
            new_response_headers.set("x-pjax-url", new_response_headers.get("x-pjax-url")
                .replace(url.protocol + "//", "https://")
                .replace(upstream_domain, url_hostname));
        }

        // Patch "location" header to handle standard 301/302/303/307/308 HTTP redirects
        if (new_response_headers.get("location")) {
            var location = new_response_headers.get("location");

            // specific behaviour for GDrive-based redirects: see https://github.com/Darkseal/CORSflare/issues/1 for details.
            if (upstream_allow_override && location.includes("googleusercontent.com")) {
                var new_upstream = location.substring(8, location.indexOf("/", 8));
                location = location + "&" + upstream_get_parameter + "=" + new_upstream;
                new_response_headers.set("location", location
                    .replace(url.protocol + "//", "https://")
                    .replace(new_upstream, url_hostname));
            }
            else {
                new_response_headers.set("location", location
                    .replace(url.protocol + "//", "https://")
                    .replace(upstream_domain, url_hostname));
            }
        }

        // Patch "set-cookie" headers by forcefully apply "SameSite=None" and "Secure" directives to allow cross-domain usage
        // (if "set_cookie_samesite_none" is set to TRUE: see that configuration option's comment block for details and references)
        if (set_cookie_samesite_none && new_response_headers.has("set-cookie")) {
            // NOTE: unfortunately the Fetch API Headers object doesn't support multiple Set-Cookie headers due to a bug in Fetch API's
            // "Headers" interface, as they are merged into a single comma-separated string (which is incompatible with most browsers).
            // ref.: https://stackoverflow.com/questions/63204093/how-to-get-set-multiple-set-cookie-response-headers-using-fetch-api
            // For that very reason, we can only support the * first * set-cookie header here.
            var firstCookie = new_response_headers.get("set-cookie").split(',').shift();
            new_response_headers.set("set-cookie", firstCookie
                .split("SameSite=Lax; Secure").join("")
                .split("SameSite=Lax").join("")
                .split("SameSite=Strict; Secure").join("")
                .split("SameSite=Strict").join("")
                .split("SameSite=None; Secure").join("")
                .split("SameSite=None").join("")
                .replace(/^;+$/g, '')
                + "; SameSite=None; Secure");
        }

        let response_content_type = new_response_headers.get('content-type');
        if (response_content_type
            && replacement_content_types.some(v => response_content_type.toLowerCase().includes(v))) {
            original_text = await replace_response_text(original_response_clone, upstream_domain, url_hostname);
        } else {
            original_text = original_response_clone.body;
        }

        response = new Response(original_text, {
            status: new_response_status,
            headers: new_response_headers
        })
    }
    return response;
}

async function replace_response_text(response, upstream_domain, host_name) {
    let text = await response.text()
    if (replacement_rules) {
        for (let k in replacement_rules) {
            var v = replacement_rules[k];

            if (replacement_use_regex) {
                k = k.replace(regexp_upstreamHostname, upstream_domain);
                k = k.replace(regexp_proxyHostname, host_name);
                v = v.replace(regexp_upstreamHostname, upstream_domain);
                v = v.replace(regexp_proxyHostname, host_name);
                text = text.replace(new RegExp(k, 'g'), v);
            }
            else {
                k = k.split('{upstream_hostname}').join(upstream_domain);
                k = k.split('{proxy_hostname}').join(host_name);
                v = v.split('{upstream_hostname}').join(upstream_domain);
                v = v.split('{proxy_hostname}').join(host_name);
                text = text.split(k).join(v);
            }
        }
    }
    return text;
}

async function is_mobile_user_agent(user_agent_info) {
    var agents = ["Android", "iPhone", "SymbianOS", "Windows Phone", "iPad", "iPod"];
    for (var v = 0; v < agents.length; v++) {
        if (user_agent_info.indexOf(agents[v]) > 0) {
            return true;
        }
    }
    return false;
}

That's very strange: your file and mine are basically identical... and mine works! You can check it out at the following URL:
https://googlesheets.ryadel.workers.dev/
Do you manage to make it work using my proxy by any chance?

Wow, it works with yours... that is indeed very strange.

OK, I think I found the bug: try to download it now, then change upstream to docs.google.com and upstream_allow_override to true (leave maybe upstream_get_parameter as it is for now).
Pay attention to NOT change upstream_get_parameter to true as you were saying in one of your replies, cause this won't work.

Thank you so much!!! And I'm sorry for causing so much trouble. 😅
It works perfectly now. 👍


mittens just reduce heat transfer

due to air pockets in it

ok past me not to break your heart but, I lost your mittens

I'll try my best to find it again, In the meanwhile just touch the 400 degree lasagna with your bare hand, Don't worry it'll grow back

oh wait that's a salamander

no non nononononononoononono  I'm not a salamender


fix: test errors due to missing window.matchMedia function

What does this PR do?
Fixes unit test errors in renderer tests.
Screenshot / video of UI

What issues does this PR fix or reference?

How to test this PR?

[ ] Tests are covering the bug fix or the new feature

we should not use hoisted pattern

@benoitf That the only pattern worked.
Please review another approach.


Entity name not matching

What version of Spook are you using?
v3.1.0
What version of Home Assistant are you using?
2024.8.2
The problem
I just installed Spook and found 12 issues. of which 11 are resolved. 1 remains and this is pretty awkward.
Spook tells me:

The raw lovelace file indeed gives me:

But going to lovelace dashboard config, it gives me:

The raw is read by spook I presume, but that is not in my dashboard config...
Anything in the logs? Paste it here!
No response

Hey homie, mind taking a look at this issue?


Update the precompiled cwebps

https://developers.google.com/speed/webp/docs/precompiled

Updated and renamed binaries so their version name is in the filename.
We now got:
cwebp-0.6.0-fbsd
cwebp-0.6.0-solaris
cwebp-0.6.1-linux-x86-64
cwebp-1.0.3-linux-x86-64
cwebp-1.0.3-linux-x86-64-static
cwebp-1.0.3-mac-10.14
cwebp-1.0.3-windows-x64.exe


Calling create-extract with embedded_datasources empty throws an error

Calling create-extract with embedded_datasources empty throws an error  'NoneType' object is not iterable
development branch
           try:
                job = server.workbooks.create_extract(workbook_item,  encrypt=args.encrypt,   includeAll=args.include_all   datasources=args.embedded_datasources, )
            except Exception as ex:
                print(ex)

output:  'NoneType' object is not iterable
fix: datasources should have a default value of []

fixed by https://github.com/tableau/server-client-python/pull/1026


Tronity API: Unexpected status: 401 (Unauthorized)

Describe the bug
When I reconnect the car after some time, evcc can no longer access the tronity api and logs the following error message:
[lp-1  ] ERROR 2024/02/17 11:42:33 vehicle soc: unexpected status: 401 (Unauthorized)

I suspect that the access token has expired and the token is not getting refetched. The expiration duration of a tronity access token is one hour.
I think the error is, that the expiration time is not correctly applied to the oauth token when the api response is getting parsed. The oauth2.Token struct tag for the Expiry field is "expiry" and not matching the response field "expires_in" of the tronity api.
I tried to fix this, but I am not sure if this is the right location to do it: https://github.com/xbaun/evcc/commit/8413183640759c8de3e7c2d37392686ec8db5997
Maybe someone can reproduce this error and verify my solution?
Steps to reproduce
When using Tronity, reconnecting the car after more than one hour should trigger the error.
Configuration details
vehicles:
  - type: template
    template: tronity
    title: TRONITY Extension
    clientid: XXXX
    clientsecret: XXXX
    capacity: 50
    name: ev5

Log details
[lp-1  ] INFO 2024/02/17 10:06:33 car connected
[lp-1  ] ERROR 2024/02/17 10:06:33 vehicle soc: unexpected status: 401 (Unauthorized)
[lp-1  ] ERROR 2024/02/17 10:06:39 vehicle odometer: unexpected status: 401 (Unauthorized)
[lp-1  ] INFO 2024/02/17 10:06:53 start charging ->
[lp-1  ] ERROR 2024/02/17 10:06:54 vehicle soc: unexpected status: 401 (Unauthorized)

What type of operating system are you running?
Linux
Version
0.124.1

think the error is, that the expiration time is not correctly applied to the oauth token when the api response is getting parsed. The oauth2.Token struct tag for the Expiry field is "expiry" and not matching the response field "expires_in" of the tronity api.

That‘s standard go and working fine. Closing due to missing trace log.

Hi @andig , thanks for your quick response.
You are right it is pretty standard oauth stuff. As far as I can oversee the token retrieval in evcc is a custom implementation when evcc token is not used.
This custom token source implementation doesn’t respect the retrieved expires_in field. When using evcc token command to retrieve a new access token, then the oauth2 implementation is used that respects the expires_in field of the response and sets the Expiry field of oauth2.Token.
While investigating this bug, I also noticed that the scopes of the tronity oauth config needs an update: https://github.com/xbaun/evcc/blob/master/vehicle/tronity/auth.go#L17
Further, the current evcc token command cannot handle a tronity vehicle template:

[main  ] FATAL 2024/02/19 09:20:32 vehicle type 'template' does not support token authentication

I would be happy to help here, but would need some help from you as to whether my thoughts are correct.
I will also provide a Trace Log when I can reproduce this behaviour the next time.

This custom token source implementation doesn’t respect the retrieved expires_in field.

You are making assumptions.

I will also provide a Trace Log when I can reproduce this behaviour the next time.

Full config and trace log please.

@andig Have you had a chance to look at the trace? I fixed it by extracting the expiry time from the tronity jwt and pass it to oauth2 token impl. Since then it runs without out the 401 failure.
Here my solution for the error I described above: https://github.com/xbaun/evcc/commit/36882aa2242a9d4593639260d12f0b8f4c3a2787#diff-fe96e494ec6bd226373c84fca0dbd17402636e4965738a601a2319bb31600f73R167
Can you reopen the issue?

@xbaun deine Lösung führt einfach dazu, dass kein token refresh gemacht wird. Wenn der dann später kommt tritt das gleiche Problem wieder auf. Anders gesagt: das ist keine Lösung :O

Duplikat: https://github.com/evcc-io/evcc/issues/12599


Localization support

Support for adding translations maybe?

Hello.
That would be great, but I only know English myself, so somebody will have to provide translations for me.
If anyone is interested in doing this, please open a PR

Well, by "Localization support" I mean you need to provide translation system. Usually with GTK software this is gettext. Take look at Resources: https://github.com/nokyan/resources, which is also Rust program.

Ah, I see, thank you for the example.
I will have a look into gettextrs and update this issue.

Localisation support should be ready in master now.

Yay!
I will try add some localizations and see how it will go.

Oh, FYI for development you will need to edit src/utils.rs and comment out lines 181 - 188, so the locale_directory is "./po"


Using notebook2script without config.ini

Summary
It would be nice to be able to use notebook2script without having to source config.ini for the destination of the python module ("lib_path").
Problem:
I've got a nbdev project set up with a config.ini file where I'm developing a library.
nbs_path = nbs
lib_path = %(lib_name)s

In the same repository, I'm creating a module with utilities that is not a part of the main lib. I would also like to
develop these utilities in notebooks and  use notebook2script to convert a utils_nbs directory to a different module of py files called utils.
As far as I understand, notebook2script does not allow for this because of line 327 in create_mod_files requires referencing config.ini for nbs_path and lib_path.
Current Workaround:
Right now I am changing my configuration file every time I want to switch which module I'm working on.
Possible Solution:
Ideally, it would be nice to have an optional parameter for notebook2script called say, lib_path, in order to specify a destination directory for the module. It seems to be that the files argument already suffices to specify a nbs_path that is different than the one specified in config.ini.
Final Thoughts:
I'd be happy to attempt a PR if this is a desirable feature. I've been using nbdev for a couple years now and I think it's the best thing since sliced bread.

I would also like to express my desire that the notebook2script also work outside the scope of a nbdev project. In other words, it would be nice to convert a standalone notebook into a script. Because there isn't any dependence on modules, this feature could even be language agnostic. This feature would require issue 536 be resolved. This is clearly a bigger ask, but hopefully at least worth flagging for the future.

@nicole-brewer this can actually be hacked super easily with the nbprocess library!
To do so, try the following from inside Jupyter:
!pip install git+https://github.com/fastai/nbprocess
from nbprocess.export import nb_export, ExportModuleProcessor
from fastcore.xtras import patch

@patch
def create_modules(self:ExportModuleProcessor):
    "Create module(s) from notebook"
    self.process()
    for mod,cells in self.modules.items():
        all_cells = self.in_all[mod]
        name = self.default_exp if mod=='#' else mod
        mm = self.mod_maker(dest=self.dest, name=name, nb_path=self.path, is_new=mod=='#')
        mm.make(cells, all_cells, lib_name=self.dest)

As this is the "hack" needed, followed by:
nb_export(nb_name, "utils")

@jph00 let me know if you think we can put this in as a PR to nbprocess 😄

What a great accidental example of why patches are so useful: clean OOP-friendly hacks 😲

@nicole-brewer I'd be happy to take a PR that adds an optional command line flag for this purpose. @muellerzr I'd be happy to look at something similar for nbprocess

nbdev now supports this feature.

Excellent, thank you!


The version number of the Modelica library is not passed to the SEMLA licence manager

For us it is very important that the version number of the Modelica library is also passed to the licence manager interface function mlle_license_new(const char *libpath,  struct mlle_error **error)
So that you can also check whether a licence is valid for a specific library version.

It is unusual to hard-connect licenses to version numbers: there are so many instances when new versions are needed. Isn't a date a much better solution in general? I agree that a version umber could be useful, but please elaborate on your use case.

in general i agree. Licenses connected to a date are the most common solution.
But we also see the need to check for the version of a library and combine it with the life time of the license.
the main use case for us is to phase out old versions (mainly old major versions).
We have made the experience that without an update obligation, outdated versions (with bugs and known issues) will still be used for many years. In this way, we minimize the support required for problems that have already been solved.

Sorry for the long delay, but I now understand your motivation. Thanks!

We don't use this ourselves, but libpath that is passed in to mlle_license_new() may contain the version, since the top level package name may include the version of the library e.g. /a/b/MyLib 1.2/.

in my opinion, this is not a good approach. Renaming the toplevel package according to the version would mean that every version each class would have to be converted when updating the library at the customer side.

We didn't mean renaming, we referred to file system naming per Modelica spec "18.8.3 Versions in the File System" but for encrypted library:

A top-level class, IDENT, with version VERSION-NUMBER can be stored in one of the following ways in a directory given in the MODELICAPATH:
• The directory IDENT with the file package.mo directly inside it
Example: Modelica/package.mo
• The directory IDENT " " VERSION-NUMBER with the file package.mo directly inside it
Example: Modelica 2.1/package.mo

But doesn't that mean that the user can manipulate the version number by renaming the folder?

No, if specified, the version number must match the version annotation in Modelica library code

@iakovn I have tested it when the library folder contains a different version number than the one in the version annoation.  But the library is still loaded.

@niklwors Could you elaborate what you tested and what would be the expected outcome?

Could you elaborate what you tested and what would be the expected outcome?

I have a library that uses a version annotation, version="1.0.1". However, the folder name contains a different version 1.0.0. I have then encrypted it with  the packagetool. But I can still load and use it in OMEdit. I have understood that this should not be possible.

@niwokr Do you mean that you start with Library 1.0.1/package.mo, encrypt that, then test "tamper" by changing the name to Library 1.0.0?
I would actually expect compiler to refuse to load that library, so something to talk about with OMEdit devs.
You are definitely able to implement the check in you own license manager, e.g., by hardcoding the version check into the LVE, right? You can also (as a workaround) specify version in a "feature" annotation.
On the default SEMLA side we currently do not include name of the top level directory into the encryption protection and we probably should. That would imply that the library decryption will fail after directory rename.

@niwokr Do you mean that you start with Library 1.0.1/package.mo, encrypt that, then test "tamper" by changing the name to Library 1.0.0?

I changed the annotation in the package.mo to 1.0.0 and used the  folder Libarary 1.0.1/ for the packagetool.

I would actually expect compiler to refuse to load that library, so something to talk about with OMEdit devs. You are definitely able to implement the check in you own license manager, e.g., by hardcoding the version check into the LVE, right?

Your suggestion was that you get the version number of the library from the  library path that is passed to the SEMLA licensemanager constructor.  That's why I asked that a user can manipulate the version number if he renames the folder.

You can also (as a workaround) specify version in a "feature" annotation.

It would be better if the version form the annoation is available in the SEMLA license manager.

On the default SEMLA side we currently do not include name of the top level directory into the encryption protection and we probably should. That would imply that the library decryption will fail after directory rename.


Windows build

coverage: 99.668%. remained the same
when pulling e52aa9d6ceca980ea090ffcd2f7b0f4943ed7ea9 on windows-build
into d996c3f8d2c7b37636bdafb6d7071a1acbc3889f on main.


wip: Disable test assembly parallelization

Attempt to resolve https://github.com/aspnet/AspNetCore-internal/issues/1531
We suspect tests hang indefinitely due to resource constraints.

Marking as wip because this is just a theory. If tests on this PR pass, i'll merge. But if we get stuck indefinitely again, we'll keep investigating.

This does not appear to resolve the issue. Closing this PR, will continue to investigate. See https://github.com/aspnet/AspNetCore-Internal/issues/1531#issuecomment-448737958


fix AccessDenied exception for write opertaion on diff threads

@ivanyu yeap I will prepare PR where i will move Permissions from RepositoryBlobContainer into StorageIO. My idea was just help developers don't think about them, but all clients behave differently, e.g. WritableByteChannel in GCS can open socket connection, Azure can create internal clients for "sync" (in reality async) operation and so on


Updated the svg and png logos with white bg.

Hi! I've updated your logos with white bg as your new favicon, using Adobe Illustrator. Hope this will be helpful! Thanks

Thank you! We should keep original logos without background too.
I suggest you to rename your newly added logo.svg to logo-white-bg.svg and logo.png to logo-white-bg.png

Sure! I will make the requested changes and commit.

Hi! I have created a new Pull Requested with the desired changes. I am closing this one. Pardon me for creating a new one for the same issue, 😅, I'm learning new things.


"make ios" and "make ios-framework" fails on osx Sierra, Xcode 8.2.1

Trace of build:
    xcodebuild -target TangramMap -project build/ios-framework/tangram.xcodeproj \
    		-configuration Release | `command -v xcpretty || echo 'xargs echo'`
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/framework/TangramMap.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/src/TGGeoPoint.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/src/TGGeoPolygon.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/src/TGGeoPolyline.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/src/TGHttpHandler.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/src/TGLabelPickResult.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/src/TGMapViewController.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    2016-12-27 15:23:46.391 xcodebuild[3832:3333236] warning:  The file reference for "ios/src/TGSceneUpdate.h" is a member of multiple groups ("Header Files" and "Resources"); this indicates a malformed project.  Only the membership in one of the groups will be preserved (but membership in targets will be unaffected).  If you want a reference to the same file in more than one group, please add another reference to the same path.
    xargs: unterminated quote
    make: *** [ios-framework] Error 1

Does anyone have have any suggestions? I only did git clone, init submodules, then tried building. make osx works perfectly.

Can you try the build again with xcpretty installed? Use gem install xcpretty to install it and then run the build again the same way.
I thought I had resolved this issue but maybe not :\

Yes - it worked perfectly after running gem install xcpretty. Please close this issue :)

Great! The build should be able to complete without xcpretty, but I'll track that in a separate issue.


ChatCompletions create() doesn't type-check enums as role

Confirm this is a feature request for the Python library and not the underlying OpenAI API.

[X] This is a feature request for the Python library

Describe the feature or improvement you're requesting
This was discussed previously @ #911, but just opening a new issue so that it's in the issue tracker.
The Bug
When calling,
role: Literal["system", "user", "assistant"] = ...
completion = await client.chat.completions.create(
    model="gpt-4",
    messages={"role": role, "content": "Hi"}
)

Reason
The main issue is that it's defined as an enum of typed dicts,
ChatCompletionMessageParam = Union[
    ChatCompletionSystemMessageParam,
    ChatCompletionUserMessageParam,
    ChatCompletionAssistantMessageParam,
    ChatCompletionToolMessageParam,
    ChatCompletionFunctionMessageParam,
]

Of course, for tooling & functions, we need to know exactly whether or not it's a user or assistant message.
Additional context
Potential Solution
From https://github.com/openai/openai-python/issues/911#issuecomment-2038669860,
I think a very simple solution would be to add another type to the ChatCompletionMessageParam union.
Currently, we have
ChatCompletionMessageParam = Union[
    ChatCompletionSystemMessageParam,
    ChatCompletionUserMessageParam,
    ChatCompletionAssistantMessageParam,
    ChatCompletionToolMessageParam,
    ChatCompletionFunctionMessageParam,
]

If instead we had,
ChatCompletionMessageParam = Union[
    ChatCompletionSystemMessageParam,
    ChatCompletionUserMessageParam,
    ChatCompletionAssistantMessageParam,
    ChatCompletionToolMessageParam,
    ChatCompletionFunctionMessageParam,
    ChatCompletionGenericMessageParam,
]

Where ChatCompletionGenericMessageParam was,
class ChatCompletionGenericMessageParam(TypedDict, total=False):
    content: Required[Optional[str]]
    role: Required[Literal["system", "user", "assistant"]]

Under my understanding, adding an option to a Union does not break compatibility with anybody (Lmk if I'm wrong)
This handles all possible situations (pydantic model, custom TypedDict with Literal, a mixture of both, etc).
And, if the user wants to pass in a tool/FunctionCall/ChatCompletionContentPartParam, then obviously they need to use the other types and prove that it's specifically a system/user/etc.
OpenAI can still internally type check safely, because it can write a type-safe function to convert ChatCompletionGenericMessageParam into a Union[ChatCompletionSystemMessageParam, ChatCompletionUserMessageParam,ChatCompletionAssistantMessageParam].

Solution is also provided at https://github.com/openai/openai-python/issues/911#issuecomment-1857142928
It's described as pydantic, but I don't think we can have message= accept both dictionaries and pydantic models. However, Message can be a function taking 2 kwargs role and user. And, it returns a Union[ChatCompletionSystemMessageParam, ChatCompletionUserMessageParam, ChatCompletionAssistantMessageParam], implemented using an if/elif/else.
Maybe name can be CreateChatCompletionMessageParam, not sure.
Just different ideas.

Thanks for the suggestion. We'll take it into consideration as we think about ways to improve this experience.


Add OSeMBE_ECEMF

Insert URLs to the project here: https://github.com/KTH-dESA/OSeMBE_ECEMF

[x] The projects is active, documented, open source licensed, shows usage from external parties and is directly targeting environmental sustainability. Find more details in the Contribution Guide.

All issues labeled as Good First Issue of the project listed on OpenSustain.tech will be visible on ClimateTriage.com. This is a great way to welcome new community members to your project.
All new projects listed will be posted on our Mastodon channel.

@HauHe @EmiFej @willu47


Movie rename functionality

Version: 2.9.11-SNAPSHOT - PRE-RELEASE
Build: 2018-05-05 07:24
OS: Linux 4.15.0-17-generic
JDK: 10.0.1 amd64 Oracle Corporation
When I use the rename feature for movies (shows untested), it does not seem to accommodate subfolders. For example, I have a large collection, so I have the "movies" folder mapped into tMM. Now, inside that folder, I have subfolders like A, B, C-D, E, etc. Inside of those folders, I have movie folders, which all contain their respective movie.
Now... when I use the rename functionality, if I have a movie such as /movies/D/Dracula/dracula.mp4, when I rename it using the feature, it is changed to /movies/Dracula/dracula.mp4. So it appears to only be able to use the 'home' folder as the mapped folder, then a subfolder for each movie. What I am trying to find is an option for the rename feature to respect the containing directory and only change the movie directory.
I did not attach logs or the specific questions asked as I do not believe they apply. This is not really a bug as it is doing what I believe it is programmed to, and my issue here is more of an oversight in functionality, or even a feature request. The only solution I can think of at this time is to map in each letter subfolder individually, which seems silly. Hope this makes sense, I will provide anything further you may want.

check renamer settings and play around with our variables....
https://www.tinymediamanager.org/blog/reworked-renamer-settingscapabitilies/

Unless I am completely missing it, I see nothing that tells it to remain in the folder that it is in, aside from a rename (potentially) of its own containing folder.

ah... you want to keep the folder?
Didn't read that out of your text....
Just empty the folder pattern should imho keep it as-is...

That may work, but then it will not do the rename I want. Here is an example of desired bahaviour:
/movies/D/Dracula/dracula movie.mp4 --> /movies/D/Dracula (1992)/dracula movie.mp4
not what it does now:
/movies/D/Dracula/dracula movie.mp4 --> /movies/Dracula (1992)/dracula movie.mp4

so you do not want to keep the folder as-is.
alright :|
Try "$1$T ($Y)" as folder?
check example afterwards?

I use the renaming function to keep the movie folders proper as shown above. I leave the movie file names as-is. I did look at the option you now suggest, and it will work only somewhat. The issue is movies that start with a period, # or a foreign character, as well as ones like stated above where the folder is C-D and not just C and D folders....
If there is no other option, I can work with what you suggest using the first letter variable, I currently just store all those (numbers, symbols, etc) starting character movies in a single folder.

tmm is designed to organize your library, so you have two options:
a) let tmm organize your folder structure
b) don't let tmm organize your folder structure
anything in between will be a half baked solution (beside that handling will be almost impossible)

I think we are all good with the above notes. I really do not see using the same 'home' folder as a "half baked solution", nor is it hard to handle as I do audio tagging / organizing all the time with such functionality without any issues. But I can see it as out of scope a bit as tMM is intended to be small/simplistic/etc. And I do agree that just letting tMM do the organizing as it wants (or with few options) is simplistic, which is in scope, and as said above, it seems to suite well enough.

sorry, didn't want to sound rude..
you have a clean folder structure where such a logic would work. But we have to respect also really messy set ups (as we've often seen before), which will probably cause problems here..
the problem here is, that the renamer is extremely complex (because we respect every setup like multi movie folder, nested folders, ...) and generating the whole folder name is one of our restrictions to work clean..
But I will still think about that in a free minute because you are the second one which requested this feature..

and what about multi movie dirs?

H \

Upload \

by me \

Movie1.avi
Movie2.avi

the existing logic would create

H \

Movie 1 \

Movie1.avi

Movie 2 \

Movie2.avi

with your setting ticket, what would you expect here? the folder by me to be renamed to movie1/2?
a folder Movie 1 to be created beneath Upload ?
these are the things we need to consider :D

What I mean is the option would be a checkbox to

retain the full path directory structure
do not retain the full path directory structure.

So, the result from above:

H \ Upload \ by me
Movie 1 \ Movie1.avi
Movie 2 \ Movie2.avi

this is as you stated above with existing logic.

with your example, you changed the path structure!
Movie 1: *H\Upload\by me* changed to *H\Upload\by me\Movie 1*
with your dracula example above that would result in:
**/movies/D/Dracula/**dracula movie.mp4 --> **/movies/D/Dracula/Dracula/**dracula movie.mp4
so to be consistent we must make sure
a) within a single movie the folder must only change the name of the last subfolder
/movies/D/Dracula/dracula movie.mp4 --> /movies/D/Dracula (1992)/dracula movie.mp4
b) within a multi movie folder we create a subfolder for each movie?
/movies/Upload/by me/Movie1.avi --> /movies/Upload/by me/Movie 1/Movie1.avi
do you aggree?

what about normalizing movies like ".45", "0815", "11:22" into a "0-9" folder when specifying $1
And/or "Æon Flux" to A (currently we take the first ASCII character -> O)
And maybe others which could not be normalized to some "#" folder?
Keeping the current movie folder, and just rename with folder/file is imho not doable.
Try renaming a pattern with subfolder (like "$1$T") multiple times.
We would always move the movie again on $1 subfolder deeper.
And i'm against of stripping all the possible subfolders from folder pattern - this always makes things unpredicable to the end user!

what about normalizing movies like ".45", "0815", "11:22" into a "0-9" folder when specifying $1?
And/or "Æon Flux" to A (currently we take the first alpha character -> O)
And maybe others which could not be normalized to some "#" folder?

All of that above is very close to what I had to start with. Sticking with your description, only difference I would have is ".45" would go in the # folder, since the period is not a number to fit into 0-9. The "Æ"would also be in the # folder (for me), but I see a strong case for its equiv to take over and use "A".
To use a more real explanation... if I write a script to search for movies, and the name of the movie is "Æon Flux"", that is what I am searching for, and Æ is a valid character, just not ASCII. So my personal logic is that the name, as-is, is a valid name, so I would have no thoughts on changing it in special character handling.
But as I said before though, using the method above that myron0815 and I ended with does work, and it does keep it simple. When I do this organizing audio (on a much larger volume), I always open the directory I want to consider 'root' and rename from there. Meaning that I can only rename the directories within that folder. This cannot apply here though... there is not directory tree browsing in tMM, and I am not so sure there would need to be.

did a small change for v2 with https://github.com/tinyMediaManager/tinyMediaManager/commit/91e586c97b96fab544a732e904eb9ae3d0274f83
If you have set the "replace ascii characters" in settings, the $1 now adheares this.
We now would get the "Æon Flux" to A, or to first latin char O.
Would not change more in v2, since this might break many existing installations...
But needs to be rethinked/changed in v3

I was thinking (on my own personal use) that another sort of solution would be to be able to have a separate rename profile for different data sources. Or maybe easier, to be able to have different rename profiles, and when you select to rename, you select which profile to use.
Example of this would be in audio taggers... you can select the format to use when auto-renaming the audio files for example.
I am only trying to think on how to get a nice improvement while still keeping it simple and easy. I am not aware of the other request for such a thing, but maybe that may help there too?

we've migrated v3 to GitLab (https://gitlab.com/tinyMediaManager/tinyMediaManager) along with all issues. The issues for v2 at GitHub will be closed - we will respond to the corresponding issue in GitLab in the future
br
Manuel


allow machines to use sendParent when they are not invoked/spawned

A machine should be able to call sendParent regardless of whether it is spawned/invoked or run on its own.
This allows greater re-usability of machines.

Also, can you please add a changeset? pray Thanks!
just checking that this was aimed at @Andarist  and not me?

Also, can you please add a changeset? pray Thanks!
just checking that this was aimed at @Andarist  and not me?

That's for you! 😉 Run yarn changeset and let's mark this as a minor change.

done :)

Could you back this feature up with a real life use case? Im really curious about this

Sure its  bit convoluted,
I was trying to get around the issue in this codesandbox https://codesandbox.io/s/sad-fog-ypdcw where by using a useSelector to access the spawned child's state, seems to be an update behind.
My fix/hack was to fire a sendParent event to the parent machine which realigned the child machines reported state.
That worked great but meant I couldn't then use my form machine unless it was a child machine.
Maybe there's a better fix for my initial problem, but in general it just seems wrong to error if a machine attempts to send an event, even if its to a non-existant parent machine.

I have a use-case for this. I have 2 machines that work fine standalone:

An auth machine handling login, logout & refreshing tokens
A separate machine which decides which content to show the user

while both work standalone, I also want to be able to compose them and expose events that a parent machine can respond to.

@davidkpiano @Andarist is there anything else you'd need from me to get this merged?

@simonflk could you illustrate this with a runnable example?
@pollen8 your particular example here should be fixed in another way. getSnapshot() used within useSelector "loses reactivity". You should first grab the formActor from the context and then subscribe to it: https://codesandbox.io/s/misty-tree-w27luf?file=/src/App.tsx:331-634

This is incredibly useful for our usecase where machines can be run as standalone, or as part of a wizard. Is there any way to get some eyes on this again, or should I create a new PR?

This is incredibly useful for our usecase where machines can be run as standalone, or as part of a wizard. Is there any way to get some eyes on this again, or should I create a new PR?

Consider my eyes on it for v5. This should go further than just failing silently when a parent doesn't exist, so here is a proof-of-concept for a dead-letter queue: https://github.com/statelyai/xstate/pull/3904

Let me give another (better) concrete example.
I have created a generic wrapper that can handle any AWS SDK client and request, with built in pagination. These machines are designed to be called standalone for simple pages, but also integrated as part of larger workflow.

However, this means that we basically need to have a factory method where our actions are defined as follows:
actions: {
  sendPageReady: settings?.sendParentEvents
    ? sendParent((context) => ({ type: 'PAGE_READY', data: context }))
    : NoOp,

  sendRequestComplete: settings?.sendParentEvents
    ? sendParent((context) => ({ type: 'REQUEST_COMPLETE', data: context }))
    : NoOp,

  sendRequestError: settings?.sendParentEvents
    ? sendParent((context) => ({ type: 'REQUEST_ERROR', data: context }))
    : NoOp,
}


packagist

Hi,
you are providing files for integration into composer, but your repository isn't listed on packagist.org , would you mind adding it?
Sure anyone could do, but if there is no service hook in your github settings, it won't update newer versions by itself.

That composer.json was part of a submitted PR (#256), but I just submitted a new package: https://packagist.org/packages/rmm5t/jquery-timeago
I also configured github webhooks so it will update automatically from here on out.


Add test for correct version string

Adds a unit test to ensure version is being picked up correctly, and is PEP 440-compliant.
The packaging lib can be used to validate version strings. It's a small library already in our dependencies indirectly though setuptools_scm, but I've added it here explicitly to our dev dependencies.

Codecov Report

Merging #363 (8a48a04) into master (c351133) will increase coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #363      +/-   ##
==========================================
+ Coverage   72.31%   72.41%   +0.09%
==========================================
  Files         157      157
  Lines       28127    28127
==========================================
+ Hits        20341    20369      +28
+ Misses       7786     7758      -28

Impacted Files
Coverage Δ

resqpy/grid.py
50.33% <0.00%> (+0.06%)
:arrow_up:

resqpy/model/_model.py
92.25% <0.00%> (+0.70%)
:arrow_up:

resqpy/model/_xml.py
88.34% <0.00%> (+2.43%)
:arrow_up:

resqpy/model/_catalogue.py
79.81% <0.00%> (+3.40%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c351133...8a48a04. Read the comment docs.


Onboarding Request: testing

Active Development
Yes
Tool name
testing
Tool abbreviation
TEST
Author(s)
Kathryn Doering
NOAA Internal
Yes
Maintainer Name
Kathryn Doering
Maintainer Email
<EMAIL_ADDRESS>Background Text
Some Text.
Citation
https://github.com/noaa-fisheries-integrated-toolbox
References
No response
Associated Tools
No response
User Organizations
No response
Online App Link
No response
Executable Link
No response
Website Link
No response
Documentation Link
No response
Source Code Link
https://github.com/noaa-fisheries-integrated-toolbox
PDF Download Link
No response
Software Badges
No response
Toolbox Drawers

[X] Ecosystems
[ ] Fish and Fisheries
[X] Human Dimensions
[ ] Protected Species
[ ] General Modeling and Productivity Tools

Keywords

[ ] ADMB
[X] Age Structured Model
[ ] Bayesian Modeling
[ ] Bycatch
[ ] C++
[X] Calculator
[ ] Data Exploration
[ ] Data Limited Method
[ ] Data Visualization
[ ] Decision Support Tool
[ ] Developer Utilities
[ ] Economics
[ ] Ecosystem
[X] Ensemble Modeling
[ ] Fish Biology
[ ] Fisheries
[ ] Gradient Solver
[ ] Guided User Interface
[ ] Individual Based Model
[ ] Legacy Tool
[ ] Length Structured Model
[ ] Management Strategy Evaluation
[ ] Marine Mammals
[ ] Maximum Likelihood Estimation
[ ] Model Builder
[ ] Model Comparision
[ ] Multispecies Model
[ ] Natural Mortality Model
[ ] NOAA Branding
[ ] North Pacific
[ ] Output Visualization
[ ] Projection Modeling
[ ] R Package
[ ] Reference Points Model
[ ] Risk Assessment Tool
[ ] Salmon
[ ] Sampling Decision Tool
[ ] Shiny
[ ] Simulation
[ ] Size Spectra Model
[ ] Size Structured Model
[ ] Socioeconomics
[ ] Spatial Model
[ ] Species Distribution Model
[ ] Standardized Reporting
[ ] Stock Assessment Model
[ ] Stock Assessment Projections
[ ] Stock Assessment Tool
[ ] Surplus Production Model
[ ] Tag Data Model
[ ] Teaching Tool
[ ] Time Series
[ ] TMB
[ ] User Utilities
[ ] Virtual Population Analysis
[ ] Visualization
[ ] Web App

Uses Github Releases
No
Static version number
No response
Comments
testing
Code of Conduct

[X] I agree to follow the FIT's Code of Conduct

Automatically generated JSON for the onboarding request:
{
  "active_development": true,
  "tool_name": "testing",
  "tool_abbreviation": "TEST",
  "authors": "Kathryn Doering",
  "noaa_internal": true,
  "maintainer_name": "Kathryn Doering",
  "maintainer_email"<EMAIL_ADDRESS>  "background_text": "Some Text.",
  "citation": "https://github.com/noaa-fisheries-integrated-toolbox",
  "source_code_link": "https://github.com/noaa-fisheries-integrated-toolbox",
  "toolbox_drawers": ["Ecosystems", "Human Dimensions"],
  "keywords": ["Age Structured Model", "Calculator", "Ensemble Modeling"],
  "uses_github_releases": false
}


gh_actions test added

github actions test integration added

Since the working principle of travis and github actions differ between each other, the test called TestBasicEnvironment passes the test, however, it sometimes fails and sometimes passes on github actions, that's why I have added ignore keyword only for github actions, so the test needs to be passed on travis CI.

Note that, seems the code_coverage_token does not work very well, I will keep fixing it, it does not effect test cases however nice to have integrated code coverage report.

I have replaced code coverage token in secrets, now it should be fine, there should not be any complain regarding to codecov report.


Added option display_week_numbers to hide week numbers in month view

Added display_week_numbers to the default options, by default it
is set to true. Then added an IF statement around the block which
displays cal-week-box

+1 I also created a pull request for this. I made it also a default option. https://github.com/Serhioromano/bootstrap-calendar/pull/470

I commented on that PR which looks good. After you change it I accept.
But this one is more complicated. This line only add week number
week.html(self.locale.week.format(p.getWeek()));

but not the whole block.
I think all you had to do is
week.html(self.locale.week.format(this.options.display_week_numbers == true ? p.getWeek() : '' ));


Fixed labelText undefined behavior

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 7 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 48.345%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/mermaid/src/dagre-wrapper/shapes/util.js
0
7
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
12624

Relevant Lines:
27159

💛  - Coveralls


Make coverage level testing a bit easier

Here's what I'm looking for

Initialize the configuration with a certain level, say 80
Make keeping that level easier with a --save-level option of something like that that would save to the configuration file the level we're interested in

That way it's easier to maintain the level, and create tests that fail if the coverage level drops. This could be the default test or a custom level command in the configuration.

Hello @JJ
As I understand, you would create rococo.ini file in the project root with content:
fail-level = 80

Alternative: run racoco as fail-level=80 racoco. I mean set environment variable.
Why that solutions do not satisfy you?

Simply make the ini file editable and configurable from the racoco command line. That way you can set up workflows that modify the fail-level when, say, you merge to main, or simply do it manually.

Sounds like J.A.R.V.I.S. from the IronMan :)
Do you know any examples of command line utilities with similar possibility? How common is this iт the command line world?

TBH, I have no idea. I know about some test runners that are able to store snapshots; jest or cypress are able to do so. This would be kinda the same thing.

Unfortunately, I think such functionality will increase a total complexity of rococo utility. As we know, the Unix principle is 'do one thing and do it well'. I already implemented a flexible properties system for a wide range of user's wishes.
I propose to leave your request open and see how many users will vote for it in the future.

I have tried to find an utility for editing ini files. There is no a common way to do so. Some uses even git config for the purpose. Probably it will be good enough for the first Raku project for someone. We need more projects like rak for Raku popularization.


Diagnostics Checks Failed. Throws the below error.

"components_agent": {
2019-06-02 07:02:00: "output": "Error executing DC/OS components health check: unable to unmarshal diagnostics response: invalid character '\u003c' looking for beginning of value\n"

dcos-diagnostics --diag --verbose
"components_agent": {
2019-06-02 07:02:00: "output": "Error executing DC/OS components health check: unable to unmarshal diagnostics response: invalid character '\u003c' looking for beginning of value\n"

Hey can you share information about version and more logs and configuration?

@schezwansoftware Are you still experiencing this issue? Please share some more info as @janisz suggested.

Similar issue noticed on DC/OS 1.11.12.
 curl <IP_ADDRESS>:61001/system/health/v1 returns 502:

<title>502 Bad Gateway</title>

502 Bad Gateway

openresty

DC/OS 1.11 is not supported please update to latest version of DC/OS


Hunter: Cat ability usage

Currently in sims Cats will only use Bite once at the start of the fight.
They should try to prioritize Bite whenever it is off cooldown.

The way we have it is how the games auto cast works for pets. If you have both abilities on autocast it will cast the higher cost one once and then never pool enough energy to cast it again.
I looked at logs and most people cast bite once and then never again due to most people not handling their pet abilities manually but leaving them on auto casta

Oh sounds like we should revert the change I made then @rosenrusinov

Im waiting to see what the community wants as both ways are valid with the second requiring a lot more attention and input

Bite is a 10s cd, while it's still hard to optimally use it most top parsers do have quite a couple of bites in their logs.
https://sod.warcraftlogs.com/reports/dJvNtnY3c7q4rHyW#fight=29&type=damage-done&source=1&ability=-883
https://sod.warcraftlogs.com/reports/1YcPTNw79dtHqy4m#fight=13&type=damage-done&source=16
https://sod.warcraftlogs.com/reports/Lt4PYK31CcnwzHZW#fight=25&type=damage-done&source=3&ability=-883


Node Engine version incorrect

I found that if you run tailor with the node engine version defined in the package (>4.4.X) you get the following error: SyntaxError: Unexpected token ... on this line: fragment.on(eventName, (...args) => {.
My colleague is running node 6.9.0 and that works fine, I am running 4.4.3 which doesn't work.

Because tailor uses ES6 features you have to run with the --harmony parameter on your version of node. https://nodejs.org/en/docs/es6/

@SimeonC tailor uses es6 features and you need harmony flag on node 4.4.x versions.
@rikkert Thanks!.

Hi, thanks for the info - sorry I didn't respond earlier!