Combine yogurt, milk, lemon juice, lemon peel, honey or agave, and basil in a blender. Blend until smooth.
Pour mixture into silicone ice cube trays. Freeze about 1 hour.
Cut paper straws into thirds. Insert straws into each mold. Freeze 4-6 hours or until firm.
To serve, gently pop out the popsicles like ice in an ice cup tray. Or for popsicle molds let set about 5 minutes before removing.