Wednesday, 25 March 2020

Track Coronavirus (COVID-19) Cases in India with Google Sheets

The Government of India website has a live dashboard that provides, in near real-time, the number of Coronavirus (COVID-19) cases in various states of India. This is the best resource to get updates around active COVID-19 cases in India.

COVID-19 Tracker for India

The official website provides the current data but if you were to check how the number of confirmed cases increased in India over time, there’s no historic data available. That’s one reason I built the COVID-19 Tracker with Google Sheets.

The tracker scrapes data from the official website every few minutes and uses Sparklines to help you visualize how the coronavirus outbreak is spreading in India over time. The Government has been actively publishing reports since March 10 and all the data can also be accessed through the Google Sheet.

Covid-19 India tracker

COVID-19 Sheets Tracker

COVID-19 JSON API

If you are a developer, I’ve also published the data as a JSON API that will provide you the latest state-wise data of COVID-19 cases as available on the Ministry of Health and Family Welfare website of India.

How the COVID-19 Tracker Works

The Coronavirus Tracker is written in Google Apps Script and it uses time-based triggers to scrape numbers from the mohfw.gov.in website every few minutes.

/**
 * Scrape the homepage of mohfw.gov.in (Ministry of Health, India)
 * website for latest numbers on Coronovirus positive cases in India
 */
const scrapeMOHWebsite = () => {
  const url = 'https://www.mohfw.gov.in/';
  const response = UrlFetchApp.fetch(url);
  const content = response.getContentText();
  return content.replace(/[\r\n]/g, '');
};

Google Apps Script doesn’t support HTML parsers like Cheerio so we had to quickly build one from scratch using regex. It grabs the HTML content of the page, looks for the table tag and then extracts data from individual cells of the table.

If they change the layout of the website, this parser is likely to break.

/**
 * Parse the webpage content and extract numbers from the HTML
 * table that contains statewise data on Covid-19 Cases in India
 */
const getCurrentCovid19Cases = (json = true) => {
  const states = {};
  const html = scrapeMOHWebsite();
  const [table] = html.match(/<div id="cases".+?>(.+)<\/div>/);
  const rows = table.match(/<tr>(.+?)<\/tr>/g);
  rows.forEach((row) => {
    const cells = row
      .match(/<td.+?>(.+?)<\/td>/g)
      .map((cell) => cell.replace(/<.+?>/g, ''));
    const [, stateName, indianNationals, foreignNationals] = cells;
    if (/[a-z\s]/i.test(stateName)) {
      states[stateName] = Number(indianNationals) + Number(foreignNationals);
    }
  });
  return json ? states : JSON.stringify(states);
};

Once we have the data in JSON format, we can easily write to a Google Spreadsheet using Apps Script. The script adds a new column per day while retaining the old data for comparison.

/**
 * Write the parsed data into a new column in Google Sheet
 * All the historic data is also preserved in the sheet.
 */
const writeNewCovid19CasesToSheets = (covid19Cases) => {
  const sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(
    'Dashboard'
  );
  const states = sheet
    .getRange(3, 1, sheet.getLastRow() - 2, 1)
    .getValues()
    .map(([state]) => [covid19Cases[state] || 0]);
  sheet
    .getRange(2, sheet.getLastColumn() + 1, states.length + 1, 1)
    .setValues([[new Date()], ...states.map((count) => [count])]);
};

The COVID-19 tracker in Google Sheets also provides a JSON API that you can use to import data directly in your apps and websites.

To publish a JSON API, we have published the script as a web app with the doGet callback function. The ContentService service returns the raw JSON output whenever an external app invokes the Google script URL.

const doGet = () => {
  const key = 'Covid19India';
  const cache = CacheService.getScriptCache();
  let data = cache.get(key);
  if (data === null) {
    data = getCurrentCovid19Cases(false);
    cache.put(key, data, 21600);
  }
  return ContentService.createTextOutput(data).setMimeType(
    ContentService.MimeType.JSON
  );
};

All the code is open-source and you are free to use in any project.



from Digital Inspiration https://ift.tt/3dvjkQA

Monday, 23 March 2020

How to Import MailChimp Subscribers to Google Sheets

The Gmail Mail Merge addon can now import the email addresses of subscribers from your MailChimp mailing lists into Google Sheets. If you wish to send emails to your subscribers directly from Gmail, instead of using MailChimp mail servers, this is the way to go.

As a developer, you can use Google Apps Script to import subscriber lists, HTML campaigns, performance reports and any other data from MailChimp to Google Sheets for analysis. You can use the MailChimp OAuth2 library but in this example, we’ll use the developer key directly to connect to MailChimp.

Get the MailChimp Developer Key

In your Mailchimp account, navigate to the Account page. In the drop-down menu, select Extras, and then API keys. Click Create A Key and make a note of it.

Google Apps Script - Get MailChimp Audiences

const MAILCHIMP_API_KEY = '<<API_KEY_HERE>>';

// MailChimp API key includes the data center id
// that your MailChimp account is associated with
const makeHttpRequest = (endpoint, params = {}) => {
  const [, mailchimpDataCenter] = MAILCHIMP_API_KEY.split('-');
  const url = `https://${mailchimpDataCenter}.api.mailchimp.com/3.0/${endpoint}`;
  const qs = Object.keys(params)
    .map(key => `${key}=${params[key]}`)
    .join('&');
  const apiUrl = qs ? `${url}?${qs}` : url;
  const request = UrlFetchApp.fetch(apiUrl, {
    method: 'GET',
    headers: {
      Authorization: `Basic ${Utilities.base64Encode(`labnol:${MAILCHIMP_API_KEY}`)}`
    }
  });
  return JSON.parse(request);
};

const getListMembers = (id, offset) => {
  const { members } = makeHttpRequest(`lists/${id}/members`, {
    count: 100,
    offset,
    fields: 'members.email_address',
    status: 'subscribed',
    sort_field: 'last_changed',
    sort_dir: 'DESC'
  });
  return members.map(({ email_address: email }) => [email]);
};

// Get a list of all subscribers of a specific
// MailChimp mailing list, you can retrieve the email address,
// name and subscription statues of subscribers
const getMailChimpListMembers = id => {
  let hasMore = true;
  let data = [];
  do {
    const emails = getListMembers(id, data.length);
    data = [...data, ...emails];
    hasMore = emails.length > 0;
  } while (hasMore);
  return data;
};

// Get a list of all audiences / lists from MailChimp
const getMailChimpLists = () => {
  const params = { count: 10, fields: 'lists.id,lists.name', sort_field: 'date_created', sort_dir: 'DESC' };
  const { lists = [] } = makeHttpRequest('lists', params);
  return lists.map(({ id, name }) => ({ id, name, members: getMailChimpListMembers(id) }));
};

The GetMailChimpLists method will bring all the lists and associated email addresses in a JSON object that you can easily write to Google Sheets using the SpreadsheetApp service.



from Digital Inspiration https://ift.tt/33GICH5

Saturday, 21 March 2020

How to Move your iCloud and Apple Photos to Google Photos

A friend writes - “Any suggestions on how to combine the Google photo library with iPhoto. Which is a better platform for keeping the photos? Google seems to be very handy in sorting and searching. Would love to know your views”

I am a big fan of Google Photos for several reasons - you get unlimited storage space, Google is pretty good at visual image search and you can have collaborative photo albums where multiple people can upload to a common folder.

Transfer Photos from Apple iCloud / Mac to Google Photos

If you would like to copy your photos from iCloud / Apple Photos to Google Photos, there are no browser plugins or software that can automate this - you’ll have to manually transfer the picture library from Apple to Google Photos. Luckily, that migration process isn’t difficult either.

It is a two-step process - you download the photos from iPhoto and iPad to your computer via iCloud and then put them on to the Google Cloud. Let’s see how:

Step 1: Download Photos from iCloud

Via Web Browser

You may directly go to the icloud.com/photos website, select your photos and then click the download button to save them from the iCloud library to your Windows PC or Mac.

Download Photos from the iCloud website

On Windows PC

If you have photos on your iPhone or iPad that are getting backed up on iCloud, you can copy them to your Windows PC with iCloud for Windows.

Inside iCloud, sign-in with your Apple ID, then open the File Explorer window, click on iCloud Photos under Quick Access and then choose Download photos and videos.

On Apple Mac OS

Open the Apple Photos app on your Mac and press ^1 (Ctrl+1) to switch to the main Photo view that has a list of all pictures that are available in your iPhoto library.

Select one or more photos, then go to the File menu and choose Export to download pictures from Apple Photos to another folder on your desktop.

Include Location in Photos

For the export options, you may stick to the default values but do check the “Include Location Information” option. This will help Google group your photos by location.

Also, if you aren’t seeing all your iPhone photos inside Mac, go to Preferences inside the Photos app and make sure that the Download originals to this Mac option is checked under the iCloud Photos section.

Download Original Photos from iCloud

Step 2 - Upload Photos to Google Photos

Once you have saved the photos that you wish to transfer to your local drive, you need to send them to Google Photos and there are two ways to go about it.

Upload Photos via Web Browser

Open Google Chrome, or any other web browser, and go to photos.google.com. Simply drag the photos folder from your desktop to the Google Photos website and they’ll be uploaded in sequence.

The browser should remain open and your Internet connection should be working for the upload to happen in the background.

Upload Photos via Google App

If you have a large number of photos to upload from your Windows PC or Mac to the Google Photos library, it is recommended that you use Google’s Backup and Sync tool that is available for both Windows and Mac OS.

Upload Apple Photos to Google Photos

With the app installed, sign-in with your Google account, select the folders on your desktop that you wish to backup and click Start to initiate the backup process.

While the name is Backup and Sync, it is not exactly a synchronization tool. So once your photos are uploaded to Google Photos, you can remove them from the local drive and they won’t be deleted from your Google Photos.



from Digital Inspiration https://ift.tt/2U9jHsH

Google Photos - The Good Parts

When Google launched Gmail in 2004, it bundled 40x more free storage space than competing webmail services. It seemed to solve all storage woes and they did not include a “delete” button in Gmail because, with a gigabyte available, why would anyone ever delete their emails. They’ve adopted a similar approach with Google Photos but gone a step further.

Google Photos offers unlimited online storage space for your digital photos and videos. The original images are compressed after uploading but the difference is barely noticeable, at least on our mobile and computer screens.

I started dumping all my pictures to Google Photos, the day it launched, and couldn’t be happier. The initial purpose was online backup but now Google Photos has become “the” place where I go to explore my photos. The files do not consume a byte of local storage space and yet the entire collection is always available on every device that I own.

Here are essential things that you should know about Google Photos and some tips to help get the most out of this amazing photo backup service.

Upload your pictures to Google Photos

Google Photos has desktop uploaders for both Windows PCs and Mac OS X. Alternatively, you can drag folders from the desktop to photos.google.com and they’ll be uploaded instantly. Android, iPhone and iPad users can install the Google Photos app and their mobile photo will be backed up automatically.

There’s no support for cloud-to-cloud transfer so if you were to move from iCloud to Google Photos, it will involve some manual effort.

Google Photos

Organize your Google Photos

Google Photos will arrange your picture library by location and by the date taken automatically. It can also recognize the visual theme of photos so if you search for, say “food” or “dinner”, you will most likely see all your family dinner photos. You can find “selfies” too. The results aren’t always accurate but a useful option nonetheless.

If you have painstakingly organized your photo on the computer in albums manually, you’ll be disappointed to know that Google Photos will ignore these albums and instead dump all the photos in one big pool. You can create photo albums inside Google Photos but it will not maintain the local album structure during upload.

Facial Recognition in Google Photos

Google Photos can recognize faces in your photos and it is pretty good at it. You can assign names to recognized faces and Google will automatically group them into albums that can be shared. Go to photos.google.com/people and you’ll find a list of all people that Google discovered in your photos.

Google Photos with Google Lens

The Google Photos app on your Android and iPhone includes Google Lens. So if you open a photo inside Photos and tap the lens icon, you can scan barcodes, business cards, recognize famous landmarks, plants and even book covers in your photos.

Duplicate Files in Google Photos

Google Photos can smartly detect duplicate photos and will skip uploading them if an image has been uploaded previously. The file names of your photos can be different and they can reside in different folders of your hard disk but the service will still recognize the duplicates and remove them from the upload queue.

There are two kinds of duplicates - exact duplicates and near duplicates. If you take a file and slightly crop it or change the EXIF data, it is a near duplicate of the original file. Google Photos will, however, treat this is a different photo and upload it as well along with the original image.

If you have too many “near duplicates” on your computer, remove the duplicates manually before adding them to the upload queue.

Deleting Files in Google Photos

You can delete a file from Google Photos and it will go to the trash. It sits there for 60 days and is then permanently removed so you have enough opportunity to restore your accidental deletes.

Here’s an important detail you should know though.

Let’s say you have a file holiday.jpg in a local Google Photos folder. If you delete this file from the Google Photos app and also empty your Google Photos’ recycle bin, the local file will get re-uploaded to Google Photos. This will happen on mobile as well. If you delete an item from Photos, the item may get re-uploaded from the phone’s gallery.

Thus, always remove files from the local folder as well after the upload else they could be re-uploaded if you ever remove the corresponding files from Google Photos.

Archive in Google Photos

Google Photos has an Instagram like Archive option that you can use for hiding photos from the main photo library. For instance, if you have photos that you don’t wish to delete but don’t want to have them in the main library, just press Shift+A and they’ll be sent to the secret archives.

Sharing with Google Photos

You can select one or more photos, hit the Share button and Google Photos will create a semi-private album with your selected photos. If you choose to share the album on, say WhatsApp, Google Photos will download and send the actual photos and not just share the link to the album.

Privacy Tip: Remember that when you share a photo or album in Google Photos via a link, anyone with that link can view your shared photos. There’s no way to limit sharing to specific email addresses as we have in Google Drive.

You can turn any photo albums in Google Photos into a common dropbox where anyone can contribute photos from their phones and desktops. So if you are a team, you can have one folder and all members can upload photos from the one to the common album.

Google Photos Animated GIF

Editing in Google Photos

Google Photos include a suite of image editing tools that let you perform basic edits and you can also apply Instagram-like filters to your images. I loved the photo editing capabilities of Google+ earlier and the same set of tools are now available in Photos. You can even produce animated GIFs, collages, and photo slideshows..

Add a Partner in Google Photos

This is a very useful feature of Google Photos - you can go to the Sharing Menu, Shared Libraries and Add a partner account (another Google Photos user). You can choose to automatically share all your photos with the partner or of specific people.

When you take a photo on your phone, it magically shows up on your partner’s phone. So if you are parent, you can choose to share photos of your kids and they’ll automatically show in your spouse’s Google account.

Move Google Photos to another Google Account

If you are switching Google Accounts, you can also move your photos from one account to another using the Partner option. Add the new Google account as a partner in your old Google Photos.

Next, sign-in to the partner account and under Shared Libraries, check the option that says Save to your Library from partner account.

If you didn’t save the photos in the new partner account from the old shared library, and your old Google Account is deleted, the photos will also be deleted from the shared library.

Google Photos - Tips & Tricks

  • You may use the Google Takeout tool to download all your Google Photos on another computer.
  • Use Chromecast to view your Google Photos as beautiful slideshows on your TV.
  • If you would like to check the status of your upload queue, go to this secret link and confirm whether files are getting added or not.
  • Google Photos are not available inside Gmail but if you have linked your Photos to Google Drive, you can easily attach any of your Google Photos in email messages.
  • Your friends can comment on photos that are shared with them. Go here to view all the comments posted on your Google Photos.

The hard disks and CDs, where you are currently storing those precious memories, will go defunct in a few years. With Google Photos now available, there’s no reason not to upload your pictures to the cloud because all you need is a decent internet connection.



from Digital Inspiration https://ift.tt/2mpEsQa

Monday, 16 March 2020

The Best Online Tools To Know Everything About a Website

How do I contact the owner of a website? Where is a particular website hosted? What other websites are hosted on that same server? Is the site using WordPress or Gatsby? Which ad networks are they using to monetize a site? Is my site accessible from China?

Here are some of the most useful online tools that will help you know every single detail of any website.

Also see: The Essentials Tools for Programmers

  • just-ping.com — Use Just Ping to determine if a particular website is accessible from other countries. Just Ping has monitoring servers across the world including Turkey, Egypt, and China so if the ping results say 100% Packet Loss, most likely the site is inaccessible from that region.
  • who.is — If you like to know the contact address, email and phone number of the website owner, this free whois lookup service will help. This is a universal lookup service meaning it can simultaneously query the whois database of all popular domain registrars.
  • whoishostingthis.com — Enter the URL of any website and this online service will show you the name of the company where that website is hosted. This may come handy if you need the contact information of the web hosting provider for writing a DMCA Notice or if you are looking to switch web hosts.
  • chillingeffects.org — When there’s a copyright-related complaint against a website, a copy of that letter is archived in the Chilling Effects database. Anyone can query this public database to know about all the copyright infringement complaints against a particular website.
  • myip.ms — MyIP.ms offers a comprehensive report of any website or I.P. Address. You get to know about the hosting provider, the physical location of a website, the IP Address change history of a website and the DNS information. Netcraft also offers similar reports.
  • reversewhois.com — The reverse whois lookup will help you determine other websites of someone. You can search the whois database by the email address or name of the domain registrant.
  • builtwith.com — Use BuiltWith to know the technology stack of any website. It helps you figure out the mail service provider of a domain, the advertising partners, the tracking widgets that are installed on a website and whether the site is using any CDN like Amazon S3 or Google Cloud. See example.
  • ssllabs.com - The certificate diagnostics tool will verify your site’s SSL certificate and ensure that it is correctly installed, trusted and does not show errors to any of your site visitors.
  • semrush.com — If you wish to analyze your competitor’s website, this is the tool to go with. SEM Rush will help you figure what organic keywords are people using to find a website, what is the site’s traffic and which are the competing websites.
  • dnsmap.io — When you buy a new domain or switch from one host to another, the DNS records for the domain changes and it may take a while to propagate these changes worldwide. The tool checks the DNS records from various geographic locations and it can check your domain’s A, CNAME, TXT and MX records. whatsmydns.net is also a good alternative.
  • toolbox.googleapps.com — If email messages, including those sent via Mail Merge, from your domain are not reaching the recipient’s mailbox, use this Google tool to confirm that DMARC, DKIM and SPF records are properly configured for your domain.
  • browserstack.com - Check your website’s responsive design on multiple desktops, tables, iOS and Android phones running different versions of operating systems.
  • screenshot.guru - If a website is inaccessible, use Screenshot Guru, hosted on the Google Cloud, to confirm if the website is down or not.
  • thinkwithgoogle.com - A mobile speed tool developed by Google that will help you determine how fast your websites will load on mobile phones on 3G and 4G network. You can also compare your mobile speed score with other websites.
  • testmysite.io - A simple site testing tool from Netlify that will measure and rank your site’s loading time from different regions around the world.
  • developers.google.com — Find the Page Speed score of any website on both desktop and mobile devices. The higher this number, the better. The Google tool also offers suggestions on how the score can be improved.
  • httparchive.org — The HTTP Archive is a repository of all performance-related metrics for a website. It keeps a record of the size of pages, their average load time and the number of failed requests (missing resources) over time.
  • Website Monitor - Use this open-source Google Sheets based website monitoring tool to get alerts when your domain goes down or is inaccessible.
  • Flush DNS - Use this tool to flush the Google DNS cache for a domain. If you changed the DNS servers for your domain, by changing registrars or DNS hosting in the last few days, flush your main domain name first before you flush any subdomains. OpenDNS also has a web tool for refreshing the DNS cache.
  • DomainTools - The tool monitors one or more web domains and sends email alerts when the domain is expiring, the domain gets renewed, the nameservers change or when the registrant information is updated for the domain.

Also see: 101 Most Useful Websites



from Digital Inspiration https://ift.tt/2lfhXww

Saturday, 7 March 2020

Block All Incoming and Outgoing Emails Except Specific Whitelisted Domains

The finance team in an organization would like to use Gmail for internal communication only. The corporate email policy restricts the finance team from sharing any files or email messages with external teams but the employees are allowed to exchange emails within the team.

Google makes it easy to implement such an email policy in Gmail for GSuite customers.

Block Incoming Emails in Gmail

To get started, sign-in to admin.google.com as your GSuite domain admin and go to Apps > GSuite Core Services > Gmail > Advanced Settings.

Inside the General Settings tab, navigate to Restrict Delivery and click the Configure button to restrict the domains that your employees are allowed to exchange emails with.

Under the Add addresses section, specify one or more domains and email addresses that employees are allowed to send and receive email messages from.

Whitelist domains and email addresses in Gmail

You can specify inputs in the following format:

  1. harvard.edu - Allow emails from everyone in the domain
  2. *.harvard.edu - Allow emails from all subdomains
  3. finance.harvard.edu - Allow emails from a specific subdomain
  4. admin@harvard.edu - Allow emails from an email address

When adding domains in the whitelist, it is recommended that you turn on sender authentication to disallow spoofed emails (where the actual sender is different from the FROM address mentioned in the email header). Gmail uses the SPF and DKIM records to verify if the sender is authenticated.

Save the settings and employees would be limited to sending emails to specific domains only.



from Digital Inspiration https://ift.tt/39y9jQh

How to Encrypt and Decrypt Text Strings with JavaScript

In one of my web projects, I required a simple encryption and decryption JavaScript library that could encode a piece of text and and then decode the encoded string on the server side.

The easiest option is the base64 encoding scheme that can be easily implemented in both native JavaScript and Google Apps Script.

Base64 Encoding with Google Apps Script

const base64Encode = text => {
  const base64data = Utilities.base64Encode(text, Utilities.Charset.UTF_8);
  return base64data;
};

const base64Decode = base64data => {
  const decoded = Utilities.base64Decode(base64data, Utilities.Charset.UTF_8);
  const input = Utilities.newBlob(decoded).getDataAsString();
  return input;
};

Base64 Encoding with JavaScript

const CryptoJS = require('crypto-js');

const encrypt = text => {
  return CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(text));
};

const decrypt = data => {
  return CryptoJS.enc.Base64.parse(data).toString(CryptoJS.enc.Utf8);
};

The obvious downside is that Base64 is encoding (not encryption) and the Base64 strings can be easily decoded.

If you are looking for a secure encryption algorithm that would require a secret passphrase for decrypting the encrypted text, go with AES. It generates longer strings but they cannot be decrypted without the password.

AES Plain Text Encryption & Decryption

const CryptoJS = require('crypto-js');

const encryptWithAES = text => {
  const passphrase = '123';
  return CryptoJS.AES.encrypt(text, passphrase).toString();
};

const decryptWithAES = ciphertext => {
  const passphrase = '123';
  const bytes = CryptoJS.AES.decrypt(ciphertext, passphrase);
  const originalText = bytes.toString(CryptoJS.enc.Utf8);
  return originalText;
};

AES Encrypt & Decryption with Google Apps Script

If you would like to use AES encryption algorithm with Google Apps Script, use the Apps Script Starter to import the CryptoJS package in your project as shown in this example.

import AES from 'crypto-js/aes';
import Utf8 from 'crypto-js/enc-utf8';

const encryptWithAES = (text, passphrase) => {
  return AES.encrypt(text, passphrase).toString();
};

const decryptWithAES = (ciphertext, passphrase) => {
  const bytes = AES.decrypt(ciphertext, passphrase);
  const originalText = bytes.toString(Utf8);
  return originalText;
};

global.testAES = () => {
  const inputText = 'Hello World';
  const passphrase = 'My Secret Passphrase';

  Logger.log({ inputText });

  const encryptedText = encryptWithAES(inputText, passphrase);
  Logger.log({ encryptedText });

  const decryptedText = decryptWithAES(encryptedText, passphrase);
  Logger.log({ decryptedText });
};

Alternatively, for Google Apps Script, the cCryptoGS library can also be used to implement AES encryption in your projects and Suite add-ons. To get started, go to Resources -> Libraries and add the MSJnPeIon6nzdLewGV60xWqi_d-phDA33 library to your Google Script project.

const encryptedMessage = cCryptoGS.CryptoJS.AES.encrypt('message', 'passphrase').toString();
Logger.log(encryptedMessage);

const decryptedMessage = cCryptoGS.CryptoJS.AES.decrypt(encryptedMessage, 'passphrase').toString(
  CryptoJS.enc.Utf8
);
Logger.log(decryptedMessage);


from Digital Inspiration https://ift.tt/38Bqhw5